RU2827404C1 - Method of transmitting bit stream over network - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to methods of encoding an image and transmitting a bit stream over a network. Method includes transmitting a bit stream over a network, where the bit stream is intended to encode a current block. According to the method, it is determined whether the previous block of coefficients, which was encoded immediately before the target block of coefficients, includes a coefficient having an absolute value greater than a threshold value. Based on the determination results, the first and second contexts to be used in arithmetic coding of the first and second flags of the target coefficient in the target coefficient block are selected. Threshold value is equal to 1.

EFFECT: reduced amount of processing without reducing efficiency of data encoding.

6 cl, 64 dwg

Description

Field of technology

[0001] The present invention relates to image encoding methods and image decoding methods.

Prior art

[0002] In recent years, with the rapid advancement of digital video equipment technology, more opportunities have emerged for compressing and encoding video signals (time sequences of moving images) and recording the resulting video signals on a recording medium such as a digital versatile disc (DVD) or a hard disk, or distributing them over a network. An example of image coding standards is H.264/AVC (MPEG-4 AVC). The High Efficiency Image Coding (HEVC) standard has recently been researched as a new generation standard (see, for example, Non-Patent Document 1).

List of links

Non-patent literature

[0003] [Non-Patent Document 1] Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 9th Meeting: Geneva, CH, 27 Apr. - 7 May 2012 JCTVC-11003_d4.doc, High efficiency video coding (HEVC) text specification draft 7 http://phenix.it-sudparis.eu/jct/doc_end_user/documents/9_Geneva/wg11/JCTVC-I1003-v5.zip

The essence of the invention

Technical task

[0004] For such an image encoding method and an image decoding method, it was necessary to reduce the amount of processing while simultaneously suppressing the deterioration of encoding efficiency.

[0005] The present invention provides an image encoding method and an image decoding method, each of which is capable of reducing the amount of processing while suppressing the deterioration of encoding efficiency.

Solution to the problem

[0006] In order to solve the above problem, according to an aspect of the present invention, a method for encoding an image using arithmetic encoding is provided, wherein the method for encoding an image includes: performing arithmetic encoding on a first flag that indicates whether an absolute value of a target coefficient in a target block of coefficients is greater than 1; and performing arithmetic encoding on a second flag that indicates whether the absolute value is greater than 2, wherein, when performing arithmetic encoding on the first flag and performing arithmetic encoding on the second flag, it is determined whether an immediately preceding block of coefficients that was encoded immediately before the target block of coefficients includes a coefficient having an absolute value greater than a threshold value, and a context to be used in arithmetic encoding on the first flag and a context to be used in arithmetic encoding on the second flag are selected based on the determination result.

[0007] These general and specific aspects may be implemented using a system, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or any combination of systems, methods, integrated circuits, computer programs, or computer-readable recording media.

Preferred effects of the invention

[0008] The present invention provides an image encoding method and an image decoding method, each of which is capable of reducing the amount of processing while suppressing the deterioration of encoding efficiency.

Brief description of the drawings

[0009]

Fig. 1 is a block diagram of an image encoding device according to Embodiment 1.

Fig. 2 is a flow chart of the sequence of operations of encoding an image according to Embodiment 1 of the image.

Fig. 3 is a block diagram of a variable-length encoding unit according to embodiment 1.

Fig. 4 is a flow chart of the variable length encoding operations according to Embodiment 1.

Fig. 5 is a block diagram of a level coding unit according to embodiment 1.

Fig. 6 is a flow chart of the sequence of operations of encoding a layer according to Embodiment 1.

Fig. 7 is a flow chart of the sequence of operations for encoding greater1_flag according to Embodiment 1.

Fig. 8 is a flow chart of the sequence of operations for encoding greater2_flag according to Embodiment 1.

Fig. 9 is a flow chart of the sequence of operations for encoding the residual according to Embodiment 1.

Fig. 10 is a flow chart of the sequence of operations for selecting a set of contexts according to embodiment 1.

Fig. 11 is a flow chart of the sequence of operations for selecting the greater1_flag context according to Embodiment 1.

Fig. 12 is a table showing an example of the numbers of contexts according to Embodiment 1.

Fig. 13 is a flow chart of the sequence of operations of binarization of the residual according to embodiment 1.

Fig. 14 is a flow chart of the sequence of operations for determining the prefix and suffix of the determination according to Embodiment 1.

Fig. 15A is a table showing an example of the correspondence relationship between a structural element (bin) of a residue, a prefix, and a suffix according to Embodiment 1.

Fig. 15B is a table showing an example of the correspondence relationship between a residue, prefix, and suffix bin according to Embodiment 1.

Fig. 15C is a table showing an example of the correspondence relationship between a residue, prefix, and suffix bin according to Embodiment 1.

Fig. 16 is a flow chart of the sequence of operations for outputting a prefix bin according to Embodiment 1.

Fig. 17 is a flow chart of the suffix bin output operation sequence according to Embodiment 1.

Fig. 18 is a flow chart of the sequence of operations for updating the binarization parameters according to embodiment 1.

Fig. 19 is a table showing the evaluation results of the image encoding method according to Embodiment 1.

Fig. 20 is a block diagram of an image decoding device according to Embodiment 2.

Fig. 21 is a flow chart of the image decoding operation sequence according to Embodiment 2.

Fig. 22 is a block diagram of a variable-length decoding unit according to embodiment 2.

Fig. 23 is a flow chart of the variable length decoding operations according to Embodiment 2.

Fig. 24 is a block diagram of a level decoding unit according to Embodiment 2.

Fig. 25 is a flow chart of the layer decoding operations according to Embodiment 2.

Fig. 26 is a flow chart of the decoding sequence of greater1_flag according to Embodiment 2.

Fig. 27 is a flow chart of the decoding sequence of greater2_flag according to Embodiment 2.

Fig. 28 is a flow chart of the residual decoding operations according to Embodiment 2.

Fig. 29 is a flow chart of the multi-valued remainder transformation operations according to Embodiment 2.

Fig. 30 is a flow chart of a prefix decoding operation sequence according to Embodiment 2.

Fig. 31 is a flow chart of the suffix decoding and prefix/suffix combining operations according to Embodiment 2.

Fig. 32A is a flow chart of the image encoding method according to Embodiment 1.

Fig. 32B is a flow chart of the first flag encoding and second flag encoding operations according to Embodiment 1.

Fig. 33A is a flow chart of the image decoding method according to Embodiment 2.

Fig. 33B is a flow chart of the first flag decoding and second flag decoding operations according to Embodiment 2.

Fig. 34 is a block diagram of a level coding unit according to Embodiment 3.

Fig. 35 is a flow chart of the layer encoding operations according to Embodiment 3.

Fig. 36 is a flow chart of the sequence of operations for selecting a set of contexts according to embodiment 3.

Fig. 37 is a table showing the evaluation results of the image encoding method according to Embodiment 3.

Fig. 38 is a block diagram of a level decoding device according to Embodiment 4.

Fig. 39 is a flow chart of the layer decoding operations according to Embodiment 4.

Fig. 40 - general configuration of the content provision system for implementing content distribution services.

Fig. 41 - General configuration of a digital broadcasting system.

Fig. 42 is a block diagram illustrating an example configuration of a television set.

Fig. 43 is a block diagram illustrating an example of a configuration of an information playback/recording unit that reads and writes information from and to a recording medium, which is an optical disc.

Fig. 44 shows an example of the configuration of a recording medium which is an optical disc.

Fig. 45A shows an example of a cellular telephone.

Fig. 45B is a block diagram showing an example configuration of a cellular telephone.

Fig. 46 illustrates the structure of multiplexed data.

Fig. 47 shows schematically how each stream is multiplexed in the multiplexed data.

Fig. 48 shows in more detail how the video stream is stored in the PES packet stream.

Fig. 49 shows the structure of TS packets and source packets in multiplexed data.

Fig. 50 shows the data structure of the RMT.

Fig. 51 shows the internal structure of multiplexed data information.

Fig. 52 shows the internal structure of the flow attribute information.

Fig. 53 shows the steps for identifying video data.

Fig. 54 shows an example of a configuration of an integrated circuit for implementing a moving image encoding method and a moving image decoding method according to each of the embodiments.

Fig. 55 shows a configuration for switching between excitation frequencies.

Fig. 56 shows the steps for identifying video data and switching between excitation frequencies.

Fig. 57 shows an example of a look-up table in which video data standards are associated with excitation frequencies.

Fig. 58A is a diagram showing an example of a configuration for sharing a signal processing unit module.

Fig. 58B is a diagram showing another example of a configuration for sharing a signal processing unit module.

Description of embodiments

[0010] Basic knowledge forming the basis of the present invention

In relation to the traditional method of image encoding, the inventors discovered the following problem.

[0011] The image encoding method according to the present HEVC standard (see, for example, Non-Patent Document 1) includes a step of predicting an encoded image, a step of calculating a difference between the predicted image and a target image, a step of converting the resulting differential image into frequency coefficients, and a step of performing arithmetic encoding on the frequency coefficients. In the arithmetic encoding, the coefficients in a block of coefficients to be encoded (coding coefficient block) are encoded sequentially in order from a coefficient of a higher frequency to a component of a lower frequency. Here, a context is selected according to a coded coefficient, and arithmetic encoding is performed on the coefficient to be encoded (coding coefficient) according to a probability of occurrence of a symbol determined based on the selected context.

[0012] In a normal image, a coefficient with a lower frequency component is likely to have a larger value. Therefore, the probability of occurrence of a symbol having an offset may be caused by determining a context with reference to a coded coefficient (a coefficient with a higher frequency than the frequency of the coding coefficient). For example, if a coded coefficient (a coefficient with a higher frequency than the frequency of the coding coefficient) has a large value, there is a high probability that the coding coefficient also has a large value. Therefore, an image coding device can reduce the resulting coding amount using the context for the large value. In the present HEVC standard, the number of coded coefficients each of which has an absolute value of 2 or more is counted, and the context for the coding coefficient is determined according to the number.

[0013] However, the inventors found that the conventional image coding method requires counting the number of coded coefficients, each of which has an absolute value of 2 or more, and therefore must have processing for counting and a resistor that stores the count value.

[0014] In order to solve the above problem, according to an aspect of the present invention, a method for encoding an image using arithmetic encoding is provided, wherein the method for encoding an image includes: performing arithmetic encoding on a first flag that indicates whether an absolute value of a target coefficient in a target block of coefficients is greater than 1; and performing arithmetic encoding on a second flag that indicates whether the absolute value is greater than 2, wherein, when performing arithmetic encoding on the first flag and performing arithmetic encoding on the second flag, it is determined whether an immediately preceding block of coefficients that was encoded immediately before the target block of coefficients includes a coefficient having an absolute value greater than a threshold value, and a context to be used in arithmetic encoding on the first flag and a context to be used in arithmetic encoding on the second flag are selected based on the determination result.

[0015] Thus, the image encoding method selects a context according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. As a result, the image encoding method can reduce the amount of processing compared to the case where the context is selected depending on the number of coefficients. In addition, the image encoding method can suppress the decrease in encoding efficiency and reduce the amount of processing.

[0016] For example, it is possible that when performing arithmetic coding on the first flag and performing arithmetic coding on the second flag, said determination is performed based on a variable generated in the arithmetic coding on the immediately preceding block of coefficients.

[0017] Thus, the image encoding method can reduce the amount of processing by performing text selection based on a variable that has been used in other processing.

[0018] For example, it is also possible that the image encoding method further includes converting into a binary representation (binarization) the residual according to a look-up table designated by the binarization parameter from among a plurality of look-up tables, when the absolute value of the target coefficient is greater than two, wherein the residual is a value obtained by subtracting 3 from the absolute value, wherein the binarization parameter is determined according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value, and wherein the variable is the binarization parameter.

[0019] For example, it is further possible that the threshold value is 1.

[0020] According to another aspect of the present invention, a method for decoding an image using arithmetic decoding is provided, wherein the method for decoding an image includes: performing arithmetic decoding on a first flag that indicates whether an absolute value of a target coefficient in a target block of coefficients is greater than 1; and performing arithmetic decoding on a second flag that indicates whether the absolute value is greater than 2, wherein, when performing arithmetic decoding on the first flag and performing arithmetic decoding on the second flag, it is determined whether an immediately preceding block of coefficients that was decoded immediately before the target block of coefficients includes a coefficient having an absolute value greater than a threshold value, and a context to be used in arithmetic decoding on the first flag and a context to be used in arithmetic decoding on the second flag are selected based on the determination result.

[0021] In this way, the image decoding method selects a context according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. As a result, the image decoding method can reduce the amount of processing compared to the case where the context is selected depending on the number of coefficients. In addition, the image decoding method can suppress the decrease in encoding efficiency and reduce the amount of processing.

[0022] For example, it is possible that when performing arithmetic decoding on the first flag and performing arithmetic decoding on the second flag, said determination is performed based on a variable generated in the arithmetic decoding on the immediately preceding block of coefficients.

[0023] In this way, the image decoding method can reduce the amount of processing by performing text selection based on a variable that has been used in other processing.

[0024] For example, it is also possible that the image decoding method further includes: converting the residual into a multi-valued value according to a look-up table designated by the binarization parameter, from among a plurality of look-up tables, when the absolute value of the target coefficient is greater than 2, wherein the residual is a value obtained by subtracting 3 from the absolute value; and determining the binarization parameter according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value, wherein the variable is the binarization parameter.

[0025] For example, it is further possible that the threshold value is 1.

[0026] According to another aspect of the present invention, an image encoding device is provided that performs arithmetic encoding, wherein the image encoding device comprises: a control circuit and a storage means accessible from the control circuit, wherein the control circuit executes the image encoding method according to claim 1.

[0027] In this case, the image decoding device selects a context in accordance with whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. As a result, the image decoding device can reduce the amount of processing compared with the case where the context is selected in accordance with the number of coefficients. In addition, the image encoding device can suppress the decrease in encoding efficiency and reduce the amount of processing.

[0028] According to another aspect of the present invention, an image decoding device is provided that performs arithmetic decoding, wherein the image decoding device includes: a control circuit and a storage means accessible from the control circuit, wherein the control circuit executes the image decoding method according to claim 5.

[0029] In this case, the image decoding device selects a context in accordance with whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. As a result, the image decoding device can reduce the amount of processing compared to the case where the context is selected in accordance with the number of coefficients. In addition, the image decoding device can suppress the decrease in encoding efficiency and reduce the amount of processing.

[0030] According to another aspect of the present invention, an image encoding and decoding device is provided, including the above-described image encoding device and image decoding device.

[0031] These general and specific aspects may be implemented using a system, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or any combination of systems, methods, integrated circuits, computer programs, or computer-readable recording media.

[0032] Next, some exemplary embodiments of the image encoding device and the image decoding device are described with reference to the accompanying drawings.

[0033] Each of the exemplary embodiments described below shows a specific example of the present invention. The numerical values, shapes, materials, structural elements, arrangement and connection of the structural elements, steps, processing order of the steps, etc. shown in the following exemplary embodiments are only examples and therefore do not limit the present invention. Therefore, among the structural elements in the following exemplary embodiments, the structural elements not mentioned in any of the independent claims are described as arbitrary structural elements.

[0034] (Implementation variant 1)

In Embodiment 1, an embodiment of an image encoding device is described.

[0035] <General Structure>

Fig. 1 is a block diagram of the structure of an image encoding device according to Embodiment 1.

[0036] The image encoding device 100 shown in Fig. 1 encodes an input image 121 to generate an encoded stream 125 (encoded bit stream). The image encoding device 100 includes an encoding block division unit 101, a subtraction unit 102, a transform unit 103, a variable-length encoding unit 104, an inverse transform unit 105, an addition unit 106, a frame memory 107, and a prediction unit 108.

[0037] <Processing (General)>

Next, the general coding flow is described with reference to Fig. 2.

[0038] (Step S101)

The coding block division unit 101 divides the input image 121 into blocks to be coded (coding blocks) 122, and sequentially supplies the coding blocks 122 to both the subtraction unit 102 and the prediction unit 108. Here, each of the coding blocks 122 has a variable size, and the coding block division unit 101 thus refers to the features of the input image 121 to divide the input image 121 into coding blocks 122. For example, one coding block 122 has a minimum horizontal 4 pixels x vertical 4 pixels and a maximum horizontal 32 pixels x vertical 32 pixels.

[0039] (Step S102)

The prediction unit 108 generates a prediction block 129 using the encoding block 122 and the decoded image 128 stored in the frame memory 107.

[0040] (Step S103)

Subtraction module 102 generates differential block 123, which is the difference between encoding block 122 and prediction block 129.

[0041] (Step S104)

The conversion module 103 converts the differential block 123 into frequency coefficients 124.

[0042] (Step S105)

The variable length encoding module 104 performs variable length encoding on the frequency coefficients 124 to generate the coded stream 125.

[0043] (Step S106)

The inverse transform module 105 transforms the frequency coefficients 124 into pixel data to reconstruct the differential block 126.

[0044] (Step S107)

The adding unit 106 adds the reconstructed differential block 126 and the predicting block 129 together to generate a decoded block 127 and stores, as a decoded image 128, the generated decoded block 127 in the frame memory 107.

[0045] (Step S108)

The processing from step S102 to step S107 is repeated until all coding blocks in the target image to be encoded are encoded.

[0046] The variable length encoding module 104 is described in more detail below.

[0047] <Structure of Variable Length Coding Module 104>

Fig. 3 is a block diagram showing the structure of the variable length encoding module 104.

[0048] In the present embodiment, the variable length encoding unit 104 encodes the target frequency coefficient 124 (hereinafter also referred to simply as the “coefficient”) based on five parameters: significant_flag, greater1_flag, greater2_flag, residual, and sign_flag.

[0049] significant_flag is a flag indicating whether the corresponding coefficient is 0. If significant_flag is 0, it means that the coefficient is 0. If significant_flag is 1, it means that the coefficient is not 0.

[0050] greater1_flag exists only when significant_flag is 1 (in other words, the coefficient is not 0). greater1_flag indicates whether the absolute value of the coefficient is 2 or greater (in other words, greater than 1). If greater1_flag is 0, it means the absolute value is 1. If greater1_flag is 1, it means the absolute value is 2 or greater.

[0051] greater2_flag exists only when greater1_flag is 1 (in other words, the coefficient has an absolute value of 2 or greater). greater2_flag indicates whether the absolute value of the coefficient is 3 or greater (in other words, greater than 2). If greater2_flag is 0, it means the absolute value is 2. If greater2_flag is 1, it means the absolute value is 3 or greater.

[0052] The remainder only exists when greater2_flag is 1 (in other words, the absolute value is 3 or greater). The remainder specifies the value obtained by subtracting 3 from the absolute value.

[0053] sign_flag exists only when significant_flag is 1 (in other words, the coefficient is not 0). sign_flag indicates whether the coefficient is negative (in other words, whether the coefficient value is negative or positive). If sign_flag is 0, it means the coefficient is positive. If sign_flag is 1, it means the coefficient is negative.

[0054] As shown in Fig. 3, the variable length encoding module 104 includes a coefficient block division module 131, a significant_flag encoding module 132, a level encoding module 133, and a sign_flag encoding module 134.

[0055] Significant_flag encoding module 132 encodes significant_flag. Level encoding module 133 encodes greater1_flag, greater2_flag, and the remainder. Sign_flag encoding module 134 encodes sign_flag.

[0056] <Processing (Variable Length Encoding)>

Next, with reference to Fig. 4, a variable-length coding stream is described.

[0057] (Step S121)

The coefficient block division unit 131 divides the encoding target block (frequency coefficients 124) into coefficient blocks, each of which has 4 pixels x 4 pixels (hereinafter also referred to as "4x4 coefficient blocks"). More specifically, when the encoding target block has a size of 32x32, the coefficient block division unit 131 divides it horizontally into 8 parts and vertically into 8 parts. When the encoding target block has a size of 4x4, the coefficient block division unit 131 does not divide such a block. The subsequent processing from step S122 to step S124 is performed for each of the coefficient blocks. The coefficient blocks are processed sequentially in order from a coefficient block having a higher component to a coefficient block having a lower component.

[0058] (Step S122)

The significant_flag encoding module 132 encodes the significant_flag of each of the coefficients included in the target coefficient block.

[0059] (Step S123)

The level encoding module 133 encodes greater1_flag, greater2_flag, and the remainder of each of the coefficients included in the target coefficient block.

[0060] (Step S124)

The sign_flag encoding module 134 encodes the sign_flag of each of the coefficients included in the target coefficient block.

[0061] (Step S125)

The processing from step S122 to step S124 is repeated until all coefficient blocks in the target encoding block have been processed.

[0062] A more detailed description of the level encoding module 133 is given below.

[0063] <Structure of Level Coding Module 133>

Fig. 5 is a block diagram showing the structure of the layer encoding module 133. As shown in Fig. 5, the layer encoding module 133 includes a greater1_flag setting module 141, a greater2_flag setting module 142, a residue setting module 143, a binarization parameter updating module 144, a context set selecting module 145, a greater1_flag memory 146, a greater1_flag context selecting module 147, a greater1_flag context memory 148, a greater2_flag context selecting module 149, a greater2_flag context memory 150, an arithmetic encoding module 151, and a residue binarization module 152. The remainder binarization module 152 includes a prefix-suffix determination module 153, a suffix 154 structural element (bin) output module 154, and a prefix bin output module 155.

[0064] The layer encoding unit 133 first selects the corresponding context sets to be used in the arithmetic encoding over greater1_flag and greater2_flag. Then, the layer encoding unit 133 encodes greater1_flag, greater2_flag and the remainder sequentially in order. Before the arithmetic encoding, the layer encoding unit 133 converts the remainder of the multi-valued signal into a binarized signal (bin). In this case, the layer encoding unit 133 adaptively changes the length of the binarized signal (bin) based on the binarization parameter (cParam). The layer encoding unit 133 also uses this binarization parameter to adaptively select the context sets.

[0065] <Processing (Level Encoding)>

Next, with reference to Fig. 6-9, the flow of encoding performed by the layer encoding unit 133 is described in more detail. Fig. 6 is a flow chart of the sequence of encoding operations performed by the layer encoding unit 133.

[0066] (Step S141)

The context set selection module 145 sets the corresponding numbers of context sets to be used in arithmetic coding over greater1_flag and greater2_flag. This step will be described in more detail below.

[0067] (Step S142)

The binarization parameter update module 144 initializes the binarization parameter (cParam) to 0. In this step, cParam is set to 0 at the beginning of the target 4x4 coefficient block. It should be noted that steps S141 and S142 are performed once for the target coefficient block. However, the following steps from step S143 are performed for each of the coefficients.

[0068] (Step S143)

The level encoding unit 133 encodes greater1_flag of each of the coefficients included in the target coefficient block. Fig. 7 is a flow chart explaining this step in more detail.

[0069] (Step S151)

The greater1_flag setting module 141 determines whether the absolute value of the target coefficient (the coefficient to be processed) is 1 or more. If the absolute value is 1 or more, greater1_flag is coded in steps S152 to S154. If the absolute value is 0, greater1_flag is not coded.

[0070] (Step S152)

If the absolute value of the target coefficient is 2 or greater, the greater1_flag setting module 141 sets greater1_flag to 1. If the absolute value is 1, the greater1_flag setting module 141 sets greater1_flag to 0.

[0071] (Step S153)

The greater1_flag context selection module 147 sets the number of contexts based on the number of context sets selected in step S141. This step will be described in more detail below.

[0072] (Step S154)

The arithmetic encoding unit 151 loads a context to be used (use context) from the greater1_flag context memory 148 based on the number of contexts selected in step S153, and then performs arithmetic encoding on greater1_flag using the context. In addition, the arithmetic encoding unit 151 stores the context that has been updated in the arithmetic encoding back to the same location in the greater1_flag context memory 148.

[0073] (Step S155)

The processing from step S151 to step S155 is repeated until all coefficients of the target 4x4 coefficient block have been processed.

[0074] As a result, greater1_flag of the corresponding coefficients are encoded in the target coefficient block.

[0075] (Step S144)

The level encoding unit 133 encodes greater2_flag of each of the coefficients included in the target coefficient block. Fig. 8 is a flow chart explaining this step in more detail.

[0076] (Step S161)

The greater2_flag setting module 142 determines whether the absolute value of the target coefficient is 2 or more. If the absolute value is 2 or more, greater2_flag is encoded in steps S162 to S165. If the absolute value is 1 or less, greater2_flag is not encoded.

[0077] (Step S162)

If the absolute value of the target coefficient is 3 or greater, then the greater2_flag setting module 142 sets greater2_flag to 1. If the absolute value is 2, then the greater2_flag setting module 142 sets greater2_flag to 0.

[0078] (Step S163)

The greater2_flag context selection module 149 sets the number of context sets selected in step S141 as the number of contexts. Unlike greater1_flag, the number of context sets for greater2_flag is directly set as the number of contexts. In other words, a single context set for greater2_flag includes only one context.

[0079] (Step S164)

The arithmetic encoding unit 151 loads a context to be used (use context) from the greater2_flag context memory 150 based on the number of contexts selected in step S150, and then performs arithmetic encoding on greater2_flag using the context. In addition, the arithmetic encoding unit 151 stores the context that has been updated in the arithmetic encoding back to the same location in the greater2_flag context memory 150.

[0080] (Step S165)

The processing from step S161 to step S164 is repeated until all coefficients of the target 4x4 coefficient block have been processed.

[0081] As a result, greater2_flag of the corresponding coefficients in the target coefficient block are encoded.

[0082] (Step S145)

The level encoding unit 133 encodes the residual in each of the coefficients in the target coefficient block. Fig. 9 is a flow chart explaining this step in more detail.

[0083] (Step S171)

The remainder setting unit 143 determines whether the absolute value of the target coefficient is 3 or more. If the absolute value is 3 or more, the remainder is encoded in steps S172 to S175. If the absolute value is 2 or less, the remainder is not encoded.

[0084] (Step S172)

The residual setting module 143 calculates a numerical value by subtracting 3 from the absolute value of the target coefficient and sets the numerical value as a residual (multi-valued signal).

[0085] (Step S173)

The residual binarization module 152 converts the multi-valued residual signal into a binarized signal. This step will be described in more detail below.

[0086] (Step S174)

The arithmetic coding module 151 performs arithmetic coding on the remainder. Unlike greater1_flag and greater2_flag, the remainder is applied with bypass arithmetic coding, which does not use any context.

[0087] (Step S175)

The binarization parameter update module 144 initializes the binarization parameter (cParam). This step will be described in more detail below.

[0088] (Step S176)

The processing from step S171 to step S175 is repeated until all coefficients in the target coefficient block have been processed.

[0089] <Processing (Selecting Context Sets)>

Next, with reference to Fig. 10, the selection of context sets (S141 in Fig. 6) will be described in more detail.

[0090] (Steps S181 to S183)

The context set selection unit 145 determines whether the target block of coefficients has the lowest frequency in the target coding block. If the target block of coefficients has the lowest frequency, the number of context sets is set to 0. Otherwise, the number of context sets is set to 2. More specifically, if the target block of coefficients is located in the upper left corner of the target coding block, the context set selection unit 145 sets the number of context sets to 0. Otherwise, the number of context sets is set to 2.

[0091] (Steps S184 and S185)

If the updated binarization parameter (cParam) of the coefficient block (the immediately preceding coefficient block) that was processed immediately before the target coefficient block is greater than 0, then the context set selection module 145 increases the number of context sets by 1. Thus, if cParam was updated at least once for the immediately preceding coefficient block, the context set is changed.

[0092] As described above, in steps S181 to S185, the number of context sets has a value in the range of 0 to 3. In other words, there are four kinds of context sets, from which the context set selection unit 145 selects one.

[0093] <Processing (Select context greater1_Flag)>

Next, with reference to Fig. 11, the selection of the greater1_flag context (S153 in Fig. 7) is described in more detail.

[0094] (Step S191)

From the number of coefficients that have been processed in the target coefficient block, the greater1_flag context selection module 147 counts the number (G1NUM) of greater1_flag with the value 1. In other words, the greater1_flag context selection module 147 counts the number of coefficients that have an absolute value of 2 or more. Here, the greater1_flag of the processed coefficients are stored in the greater1_flag memory 146, so that the greater1_flag context selection module 147 obtains these greater1_flag from the greater1_flag memory 146.

[0095] (Steps S192 to S193)

If G1NUM calculated in step S191 is greater than or equal to 1, the greater1_flag context selection unit 147 sets the context offset to 3. If G1NUM is 0, the processing proceeds to step S194. In other words, when there is at least one processed coefficient having an absolute value of 2 or more in the target coefficient block, the context offset is set to 3.

[0096] (Step S194)

From the processed coefficients in the target coefficient block, the greater1_flag context selection module 147 counts the number (G1NUM2) of coefficients having greater1_flag. In other words, the greater1_flag context selection module 147 counts the number of coefficients having an absolute value of 1 or greater. Here, the greater1_flag context selection module 147 obtains greater1_flag of processed coefficients from the geater1_flag memory 146.

[0097] (Stages S195 to S197)

If G1NUM2 calculated in step S194 is greater than 2, the greater1_flag context selection module 147 sets the context offset to 2. Otherwise, the greater1_flag context selection module 147 sets the context offset to the value of G1NUM2.

[0098] Thus, in steps S191 to S197, the context offset is set to a value in the range of 0 to 3.

[0099] (Step S198)

The greater1_flag context selection module 147 sets the number of contexts to "(number of context sets × 4) + context offset" and then the resulting number of contexts into the greater1_flag context memory 148. Since the number of context sets has a value from 0 to 3, and the context offset has a value from 0 to 3, as shown in Fig. 12, the number of contexts ultimately has a value in the range from 0 to 15. In other words, there are four kinds of context sets, and there are four contexts in each of the context sets for greater1_flag. The context set selection module 145 selects one kind of context sets from the four kinds, and the greater1_flag context selection module 147 selects one of the four contexts in the selected context set.

[0100] <Processing (Residual Binarization)>

Next, with reference to Fig. 13-16, the binarization of the remainder (S173 in Fig. 9) is described in more detail. Thus, the binarized signal (bin) includes a prefix and a suffix. The method for determining the prefix and the suffix is changed according to the binarization parameter (cParam).

[0101] (Step S201)

The prefix-suffix determination module 153 determines a prefix and a suffix. Fig. 14 is a flow chart explaining this step in more detail.

[0102] (Steps S211 to S222)

The prefix-suffix determination module 153 uses cParam in determining the prefix and suffix. The determination processing is mainly classified into the following two methods. If the remainder is small, the prefix and suffix are determined in steps S213 and S214. On the other hand, if the remainder is large, the prefix and suffix are determined in steps S215 to S222. cParam is used as a threshold value based on which it is determined whether the remainder is small or large. As cParam is larger, the threshold value is larger. cParam is related to the bin length of the prefix and suffix.

[0103] Figs. 15A-15C show a prefix and suffix bin in association with a cParam and a residue. As shown in Figs. 15A-15C, if the residue is small, a smaller cParam results in a shorter bin length, and if the residue is larger, a larger cParam results in a shorter bin length. In other words, if the residue were small, a smaller cParam would be more likely to improve coding efficiency. On the other hand, if the residue were large, a larger cParam would be more likely to improve coding efficiency.

[0104] (Step S202)

The prefix bin output module 155 provides the prefix bin to the arithmetic coding module 151. Fig. 16 is a flow chart explaining this step in more detail.

[0105] (Steps S231 to S235)

The prefix bin output module 155 first outputs "1" values in the same quantity as the prefix value, and finally outputs "0" values.

[0106] (Step S203)

The suffix bin output module 154 provides the suffix bin to the arithmetic coding module 151. Fig. 17 is a flow chart explaining this step in more detail.

[0107] (Steps S241 to S245)

The suffix bin output module 154 converts the suffix value into a binary-coded form and outputs the resulting values in binary-coded form sequentially in order from the highest-order bit. The suffix bin output module 154 outputs bin(s) in the quantity tmpLen. In this case, if the number of bits in the suffix value is less than tmpLen, the highest-order bit is output as 0.

[0108] <Processing (Update Binarization Parameter)>

Next, with reference to Fig. 18, the updating of the binarization parameter (S175 in Fig. 9) is described in more detail.

[0109] (Step S251)

If the binarization parameter (cParam) is less than 4, the binarization parameter updating unit 144 updates cParam in steps S252 to S254. If cParam is 4 or greater, the binarization parameter updating unit 144 does not update cParam and ends the processing.

[0110] (Steps S252 to S254)

If the absolute value of the target coefficient is greater than the threshold value, the binarization parameter update module 144 increases cParam by 1. The threshold value is determined using the mathematical formula "3x(1<< cParam)". "<<" means left shift. When cParam has a greater value, the threshold value has a greater magnitude. As described above, cParam is initialized to 0 at the beginning of processing for the target block of coefficients. Each time a coefficient greater than the threshold value occurs due to the processing of the binarization parameter update module 144, cParam is increased by 1 until cParam reaches a maximum value of 4.

[0111] <Effects>

As described above, in the image encoding device 100 according to the present embodiment, if there is a coefficient having an absolute value greater than a threshold value in the target processed coefficient block, the context for greater1_flag and the context for greater2_flag are changed. As a result, encoding efficiency can be improved with a small amount of processing.

[0112] More specifically, the binarization parameter (cParam) is increased by 1 each time an absolute value of a coefficient greater than a threshold value is found. If, at the beginning of the target coefficient block, the value of cParam that was updated in the processing of the immediately preceding coefficient block is greater than 0, in other words, if the immediately preceding coefficient block includes at least one coefficient greater than a threshold value, then the number of context sets is changed. More specifically, if there is a coefficient having a large value in the immediately preceding coefficient block (a coefficient block with a higher frequency than the frequency of the target coefficient block), then the image encoding device 100 determines that there is a high probability that the target coefficient block also includes a coefficient having a large value. Therefore, a context suitable for the coefficient having a large value is selected.

[0113] In the present HVEC standard (see Non-Patent Document 1), the number of processed coefficients having an absolute value of 2 or more is counted, and the context for the target coefficient is determined according to the number. However, the processed coefficients are not limited to the coefficients included in the immediately preceding coefficient block. They may be a plurality of coefficients included in a plurality of coefficient blocks that are located closer to the high-frequency side than the target coefficient block.

[0114] On the other hand, in the method according to the present embodiment, it is not determined whether or not there is a coefficient exceeding a threshold value in the immediately preceding block of coefficients. Thus, the image encoding device 100 according to the present embodiment does not need to count the number nor have a resistor for storing the count values.

[0115] Furthermore, the image encoding method according to the present embodiment was applied to test the software of the present HEVC standard for experiments. Fig. 19 shows the experimental results compared with the state before application. The experimental conditions are based on the general experimental conditions of the HEVC standard group. A larger numerical value indicates lower encoding efficiency. A negative value indicates an improvement in encoding efficiency. As shown in Fig. 19, each of the values is in the range of -0.01% to 0.03%. This means that the encoding efficiency is hardly affected even if the coefficient counting is eliminated.

[0116] In addition, by combining information to be used in selecting contexts and information to be used in encoding the residual, the size of the circuit is reduced. More specifically, cParam, which is incremented each time an absolute value of a coefficient that is greater than a threshold value is found, is used to change the binarization parameter of the residual and to select the greater1_flag context and the greater2_flag context, so that one structure can perform various functions. Thus, compared with the existing HEVC standard, the image encoding method according to the present embodiment can eliminate the coefficient counting, while the deterioration in encoding efficiency is suppressed, without adding a new structure.

[0117] It should be noted that it has been described above that the context set is changed based on the binarization parameter (cParam) that has been updated for the immediately preceding coefficient block. However, the image encoding device 100 can perform the changes based on the binarization parameter that has been updated for a coefficient block other than the immediately preceding coefficient block. It is possible, for example, that the image encoding device 100 can change the context set when cParam exceeds 0 in the processing for any coefficient block being processed. The coefficient blocks are processed sequentially in order from the higher-frequency block. Thus, if cParam of any one of the coefficient blocks exceeds 0, in other words, if a coefficient appears that has an absolute value that is greater than or equal to the threshold value, there is a high probability that all subsequent coefficient blocks have larger coefficient values. In such a case, it is possible to increase the coding efficiency by selecting a context set to be used for the above possibility.

[0118] In addition, the image encoding device 100 initializes the binarization parameter (cParam) to 0 at the beginning of the processing for the target coefficient block. However, it is also possible to support the use of the cParam updated for the immediately preceding coefficient block without initializing cParam to 0. This makes it possible to continue to use the state of the high-frequency coefficient block. As a result, the encoding efficiency is sometimes increased depending on the input image.

[0119] It should also be noted that it has been described above that the image encoding device 100 changes the context set when cParam is greater than 0. However, the present embodiment is not limited to the above. It is also possible to change the context set if, for example, cParam is greater than 1, or cParam is greater than 2. Depending on the input image, it is sometimes possible to improve encoding efficiency by changing the context set when cParam is greater than 1.

[0120] It should also be noted that it has been described above that four kinds of context sets are used, but the present embodiment is not limited to the above. There may be four or more kinds of context sets, or four or less kinds of context sets. For example, although it has been described above that the image encoding device 100 increases the number of context sets by 1 when cParam is greater than 0, when the number of context sets is 0 or 2. However, it is also possible that the number of context sets is increased by 1 only when the number of context sets is 2.

[0121] It should also be noted that it has been described above that the image encoding device 100 changes the context set when the target coefficient block has the lowest frequency, but the change may be unnecessary. It is also possible that the image encoding device 100 increases the number of context sets by 1 when cParam is greater than 0, and further increases the resulting number of context sets by 1 when cParam is greater than 1. Depending on the input image, the encoding efficiency can be increased when the number of kinds of context sets increases. On the other hand, depending on the input image, when the number of context sets decreases, it is possible to reduce the kinds of contexts, thereby reducing the size of the circuit for processing the selection and reducing the memory for storing contexts.

[0122] It should also be noted that in Fig. 11, it was shown that the image encoding device 100 sets the greater1_flag context based on G1NUM and G1NUM2, but the present embodiment is not limited to the above. For example, the image encoding device 100 may set the context based on the position of the target coefficient in the target coefficient block, or based on G1NUM or G1NUM2.

[0123] It should also be noted that it has been described above that when selecting a context for greater2_flag, the image encoding device 100 sets the number of context sets as the actual number of context, but the present embodiment is not limited to the above. The image encoding device 100 can select a context for greater2_flag in the same way as used for greater1_flag.

[0124] It should also be noted that it has been described above that greater1_flag exists when the absolute value of the target coefficient is 1 or greater, and that greater2_flag exists when the absolute value of the target coefficient is 2 or greater. However, it is also possible to limit the corresponding maximum number of flags in the coefficient block in the same way as in the present HEVC standard (Non-Patent Document 1). It is possible that there are a maximum of eight greater1_flag and a maximum of one greater2_flag.

[0125] It should also be noted that it has been described above that in the binarization of the residual of the image encoding device 100, the prefix and the suffix form a bin. However, the binarized residual signal may be any signal as long as the bin length can be changed based on cParam. For example, the binarized residual signal may have only a prefix or may be expressed using the Golomb exponent method of the H.264 moving picture encoding standard.

[0126] It should also be noted that it was described above that the last bin of the prefix is set to 0, but the last 0 can be removed when the prefix is the maximum. If the prefix is not the maximum, it is necessary to insert "0" to clarify the bin at the boundary of the prefix and the suffix. However, if the image decoding device knows the maximum value of the coefficient, the maximum value of the prefix is obvious. Thus, if the prefix is the maximum, the image decoding device can determine the boundary between the prefix and the suffix without "0". Eliminating the insertion of "0" in the case of the maximum prefix can significantly shorten the bin length when there are a large number of coefficients with the maximum prefix (in other words, when the coding bit rate is high). As a result, the coding efficiency is improved.

[0127] It has been described above that the threshold value used in updating the binarization parameter (cParam) is set according to the mathematical formula "3*(1<<cParam)", but the present embodiment is not limited to the above. For example, the image encoding device may use another numerical value, such as "4" instead of "3" in the mathematical formula, or use another mathematical formula.

[0128] It should also be noted that it has been described above that the image encoding device uses a context in the arithmetic coding over greater1_flag and greater2_flag for all coefficients, but it is also possible that a bypass arithmetic coding that does not use a context can be performed on the greater1_flag or greater2_flag of a part of the coefficients in the same way as it is performed on the remainder. For example, if greater1_flag of the immediately preceding coefficient is 1, the image encoding device can perform a bypass arithmetic coding on the greater1_flag of the target coefficient. It is also possible that the image encoding device performs a bypass arithmetic coding on greater1_flag at the beginning of a block of target coefficients. Using a bypass arithmetic coding can eliminate loading and updating the context. In addition, using a bypass arithmetic coding can eliminate a dependency relationship with another syntax using the same context. As a result, the processing can be performed at a higher speed.

[0129] It should also be noted that it was described above that the coding block has a size of a maximum of 32 x 32 pixels and a minimum of 4 x 4 pixels, but the size of the coding blocks is not limited to the above. It is also possible that the coding blocks have a fixed size.

[0130] It should also be noted that it was described above that the coding block is divided into 4×4 coefficient blocks, but it is not necessary to divide the coding block into coefficient blocks, each of which has 4 pixels×4 pixels. For example, it is also possible to divide the coding block into coefficient blocks, each of which has 8 pixels×8 pixels. The coding block can be divided into rectangular coefficient blocks, each of which has, for example, 8 pixels×4 pixels.

[0131] Furthermore, the processing according to the present embodiment may be implemented in software. This software may be distributed by downloading or the like. Or, this software may be recorded on a recording medium such as a CD-ROM for the purpose of distribution. The above applies to other embodiments in this description.

[0132] (Embodiment 2)

In Embodiment 2, a description is given of an image decoding device that decodes an encoded bit stream generated by the image encoding device according to Embodiment 1.

[0133] <General Structure>

Fig. 20 is a block diagram showing the structure of an image decoding device 200 according to the present embodiment.

[0134] The image decoding device 200 shown in Fig. 20 decodes the coded stream 221 to generate a decoded image 225. Here, the coded stream 221 corresponds to, for example, the coded stream 125 generated by the above-described image encoding device 100. The image decoding device 200 includes a variable-length decoding unit 201, an inverse transform unit 202, an adding unit 203, a decoded block combining unit 204, and a frame memory 205.

[0135] <Processing (General)>

Next, with reference to Fig. 21, the general decoding flow is described.

[0136] (Step S301)

The variable length decoding module 201 performs variable length decoding on the coded stream 221 to generate frequency coefficients 222 and provides the frequency coefficients 222 to the inverse transform module 202.

[0137] (Step S302)

The inverse transform module 202 transforms the coefficients 222 of the frequency 222 into pixel data to generate a differential block 223.

[0138] (Step S303)

The summing unit 203 sums the decoded image 226 stored in the frame memory 205 with the differential block 223, thereby generating a decoded block 224.

[0139] (Step S304)

The processing from step S301 to step 303 is repeated until all blocks in the target image have been decoded.

[0140] (Step S305)

The decoded block combining unit 204 combines the plurality of decoded blocks 224 together to generate a decoded image 225 and stores, as a decoded image 226, the decoded image 225 in the frame memory 205.

[0141] A more detailed description of the variable length decoding module 201 is given below.

[0142] <Structure of Variable Length Decoding Module 201>

Fig. 22 is a block diagram showing the structure of the variable-length decoding unit 201. In the present embodiment, similar to Embodiment 1, the frequency coefficient 222 (hereinafter also referred to simply as "coefficient") is expressed based on five parameters: significant_flag, greater1_flag, greater2_flag, remainder, and sign_flag. The corresponding parameters indicate the same as those described in Embodiment 1, so that repeated explanation is not given.

[0143] As shown in Fig. 22, the variable length decoding module 201 includes a significant_flag decoding module 231, a level decoding module 232, a sign_flag decoding module 233, and a coefficient decoding module 234.

[0144] <Processing (Variable Length Decoding)>

Next, with reference to FIG. 23, a variable word length decoding flow is described.

[0145] (Step S321)

The significant_flag decoding module 231 decodes the significant_flag of each of the coefficients in the target coefficient block and provides the resulting significant_flag to the level decoding module 232, the sign_flag decoding module 234, and the coefficient decoding module 234.

[0146] (Step S322)

The level decoding module 232 decodes greater1_flag, greater2_flag, and the remainder of each of the coefficients in the target coefficient block and provides the resulting greater1_flag, greater2_flag, and the remainder to the coefficient decoding module 234. Here, the level decoding module 232 decodes greater1_flag, greater2_flag, and the remainder only when significant_flag is equal to 1.

[0147] (Step S323)

The sign_flag decoding module 233 decodes the sign_flag of each of the coefficients in the target coefficient block and provides the resulting sign_flag to the coefficient decoding module 234. Here, the sign_flag decoding module 233 decodes the sign_flag only when significant_flag is equal to 1.

[0148] (Step S324)

The coefficient decoding unit 234 decodes the target coefficient based on significant_flag, greater1_flag, greater2_flag, the residual, and sign_flag. The corresponding parameters indicate the same as described in Embodiment 1, so that the coefficient decoding unit 234 decodes the target coefficient in accordance with the indication of the parameters.

[0149] A more detailed description of the level decoding module 232 is given below.

[0150] <Structure of Level Decoding Module 232>

Fig. 24 is a block diagram showing the structure of the layer decoding module 232. As shown in Fig. 24, the layer decoding module 232 includes a context set selection module 241, a greater1_flag context selection module 242, a greater1_flag memory 243, a greater1_flag context memory 244, a greater2_flag context selection module 245, a greater2_flag context memory 246, an arithmetic decoding module 247, a binarization parameter updating module 248, and a multi-valued residue module 249. The multi-valued residue module 249 includes a prefix decoding module 250, a suffix decoding module 251, and a prefix-suffix combining module 252.

[0151] <Processing (Level Decoding)>

Next, with reference to Fig. 25-28, the level decoding will be described in more detail. Fig. 25 is a flow chart of the decoding sequence performed by the level decoding module 232.

[0152] (Step S341)

The context set selection module 241 sets the respective numbers of context sets to be used in the arithmetic decoding over greater1_flag and the arithmetic decoding over greater2_flag. The setting method is the same as in the context set selection (FIG. 10) according to Embodiment 1.

[0153] (Step S342)

The binarization parameter updating module 248 initializes the binarization parameter (cParam) to 0. In this step, cParam is set to 0 at the beginning of the target coefficient block. It should be noted that steps S341 and S342 are performed once for the target coefficient block, while the following steps from step S343 are performed for each of the coefficients in the target coefficient block.

[0154] (Step S343)

The level decoding module 232 decodes greater1_flag of each of the coefficients in the target coefficient block. Fig. 26 is a block diagram explaining this step in more detail.

[0155] (Step S351)

The level decoding unit 232 determines whether significant_flag of the target coefficient is equal to 1. If significant_flag is equal to 1, the level decoding unit 232 decodes greater1_flag in steps S352 and S353. On the other hand, if significant_flag is equal to 0, the level decoding unit 232 does not decode greater1_flag.

[0156] (Step S352)

The greater1_flag context selection module 242 sets the number of contexts based on the number of context sets selected in step S341. The setting method is the same as in the greater1_flag context selection (Fig. 11) according to Embodiment 1.

[0157] (Step S353)

The arithmetic decoding unit 247 loads a context to be used (use context) from the greater1_flag context memory 244 according to the number of contexts selected in step S352, and performs arithmetic decoding on greater1_flag by using the context. In addition, the arithmetic decoding unit 247 stores the context that was updated in the arithmetic decoding back to the same location in the greater1_flag context memory 244.

[0158] (Step S354)

The processing from step S351 to step S353 is repeated until all coefficients in the target coefficient block have been processed.

[0159] As a result, greater1_flag of the corresponding coefficients in the target coefficient block are decoded.

[0160] (Step S344)

The level decoding module 232 decodes greater2_flag of each of the coefficients in the target coefficient block. Fig. 27 is a block diagram explaining this step in more detail.

[0161] (Step S361)

The level decoding unit 232 determines whether greater1_flag of the target coefficient is equal to 1. If greater1_flag is equal to 1, the level decoding unit 232 decodes greater2_flag in steps S362 and S363. On the other hand, if greater1_flag is equal to 0, the level decoding unit 232 does not decode greater2_flag. It should be noted that if significant_flag of the target coefficient is equal to 0, the level decoding unit 232 decodes neither greater1_flag nor greater2_flag.

[0162] (Step S362)

The greater2_flag context selection module 245 sets the number of context sets selected in step S341 as the number of contexts. Unlike greater1_flag, the number of context sets for greater2_flag is directly set to the number of contexts. In other words, a single context set for greater2_flag includes only one context.

[0163] (Step S363)

The arithmetic decoding unit 247 loads a context to be used (use context) from the greater2_flag context memory 246 according to the number of contexts selected in step S362, and performs arithmetic decoding of greater2_flag by using the context. In addition, the arithmetic decoding unit 247 stores the context that was updated in the arithmetic decoding back to the same location in the greater2_flag context memory 246.

[0164] (Step S364)

The processing from step S361 to step S363 is repeated until all coefficients in the target coefficient block have been processed.

[0165] As a result, greater2_flag corresponding coefficients in the target coefficient block are decoded.

[0166] (Step S345)

The level decoding unit 232 decodes the remainder of each of the coefficients in the target coefficient block. Fig. 28 is a flow chart explaining this step in more detail.

[0167] (Step S371)

The layer decoding unit 232 determines whether greater2_flag of the target coefficient is equal to 1. If greater2_flag is equal to 1, the layer decoding unit 232 decodes the residual in steps S372 and S373. On the other hand, if greater2_flag is equal to 0, the layer decoding unit 232 does not decode the residual. It should be noted that if significant_flag of the target coefficient is equal to 0 or greater1_flag of the target coefficient is equal to 0, the layer decoding unit 232 does not decode either greater2_flag or the residual.

[0168] (Step S372)

Arithmetic decoding module 247 performs arithmetic decoding of the remainder. Unlike greater1_flag and greater2_flag, the remainder is applied with bypass arithmetic coding, which does not use any context.

[0169] (Step S373)

The multi-valued residue module 249 converts the multi-valued residue signal into a binarized signal. This step will be described in more detail below.

[0170] (Step S374)

The binarization parameter updating module 248 updates the binarization parameter (cParam). The updating method is the same as the updating of the binarization parameter (FIG. 18) according to embodiment 1.

[0171] (Step S375)

The processing from step S371 to step S374 is repeated until all coefficients in the target coefficient block have been processed.

[0172] <Processing (Converting Residual to Multi-Valued Signal)>

Next, with reference to Figs. 29-31, the transformation of the remainder into a multi-valued signal (S373 in Fig. 28) is described in more detail.

[0173] (Step S401)

The prefix decoding module 250 decodes the prefix. Fig. 30 is a flow chart explaining this step in more detail.

[0174] (Steps S411 to S415)

The prefix decoding module 250 determines the prefix. Finally, the prefix decoding module 250 receives codes of 1 bit each sequentially from the arithmetic decoding module 247 until "0" is found. The number of continuous "1"s is set to the prefix value.

[0175] (Step S402)

The suffix decoding module 251 decodes the suffix.

[0176] (Step S403)

The prefix-suffix combining module 252 combines the prefix and the suffix to generate a multi-valued remainder signal.

[0177] Fig. 31 is a flow chart explaining steps S402 and S403 in more detail.

[0178] (Steps S421 to S433)

The suffix decoding unit 251 determines the suffix. Finally, if the prefix is less than 8, the suffix decoding unit 251 sets the value of cParam as the bin length and generates the suffix by ordering the results of arithmetic decoding of the bin length in order from a higher order. On the other hand, if the prefix is 8 or greater, the suffix decoding unit 251 calculates the bin length from the prefix and cParam and generates the suffix by ordering the results of arithmetic decoding of the bin length in order from a higher order.

[0179] (Steps S441 to S442)

The prefix-suffix combining module 252 computes the remainder based on the prefix, suffix, and cParam.

[0180] <Effects>

As described above, the image decoding device according to Embodiment 2 can provide the same effects as described in Embodiment 1.

[0181] Fig. 32A is a flow chart of the sequence of image encoding operations performed by the above-described image encoding apparatus 100.

[0182] As described above, the image encoding device 100 performs arithmetic encoding on a first flag (greater1_flag) indicating whether the absolute value of the target coefficient included in the target coefficient block is greater than 1 (S701). Next, the image encoding device 100 performs arithmetic encoding on a second flag (greater2_flag) indicating whether the absolute value is greater than 2 (S702).

[0183] Fig. 32B is a flow chart of steps S701 and S702.

[0184] In steps S701 and S702, the image encoding device 100 determines whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value (S711), and, based on the determination result, selects corresponding contexts to be used in the arithmetic encoding on the first flag and the arithmetic encoding on the second flag (S712). In other words, the image encoding device 100 makes the above determination with reference only to the coefficients included in the immediately preceding block of coefficients, and not to the coefficients included in the coefficient blocks other than the immediately preceding block of coefficients. In other words, the image encoding device 100 performs arithmetic encoding on the first flag and the second flag using the contexts corresponding to the determination result from among the plurality of contexts.

[0185] The immediately preceding block of coefficients refers to a block of coefficients that were processed immediately before the target block of coefficients.

[0186] More specifically, if the absolute value of the target coefficient is greater than the threshold value, the image encoding device 100 increments the binarization parameter (cParam) (S253 and S254 in Fig. 18). Then, if cParam from the immediately preceding block of coefficients is greater than 0, the image encoding device 100 changes the context (in other words, increments the number of context sets) (S184 and S185 in Fig. 10).

[0187] In other words, based on a variable generated in arithmetic coding over an immediately preceding block of coefficients, the image coding device 100 determines whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value.

[0188] More specifically, if the absolute value of the target coefficient is greater than 2, the image encoding device 100 binarizes the remainder, that is, the value obtained by subtracting 3 from the absolute value, according to a lookup table designated by a binarization parameter (cParam) among a plurality of lookup tables. This binarization parameter is determined depending on whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. The above-mentioned variable is this binarization parameter.

[0189] It has been described above that the threshold value for the absolute values of the coefficients, which is referred to when changing the context, is determined based on the binarization parameter (S252 in Fig. 18), but the threshold value may be a predetermined value ("1") as described with respect to the conventional technology. In other words, the image encoding device 100 can determine whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than 1 (2 or more), and change the context to be used to another one based on the determination result. As described above, in the conventional method, the number of coefficients having an absolute value greater than the threshold value is counted. This processing is different from the processing of determining whether there is a coefficient having an absolute value greater than the threshold value as described in the present embodiment.

[0190] In this case, as in the previous case, the image encoding device 100 may determine whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than 1, based on a variable generated in arithmetic coding over the immediately preceding block of coefficients. For example, this variable is greater1_flag. In other words, the image encoding device 100 may change the context according to whether the immediately preceding block of coefficients includes a coefficient that has greater1_flag having a value equal to 1.

[0191] Fig. 33A is a flow chart of the sequence of image decoding operations performed by the above-described image decoding device 200.

[0192] As shown in Fig. 33A, the image decoding device 200 performs arithmetic decoding on a first flag (greater1_flag) that indicates whether the absolute value of the target coefficient included in the target coefficient block is greater than 1 (S751). The image decoding device 200 performs arithmetic decoding on a second flag (greater2_flag) that indicates whether the absolute value is greater than 2 (S752).

[0193] Fig. 33B is a flow chart of steps S751 and S752.

[0194] In steps S751 and S752, the image decoding device 200 determines whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value (S761), and, based on the determination result, changes a corresponding context to be used in arithmetic decoding on the first flag and the second flag (S762). In other words, the image decoding device 200 performs arithmetic decoding on the first flag and the second flag by using corresponding contexts corresponding to the determination result, among a plurality of contexts.

[0195] More specifically, if the absolute value of the target coefficient is greater than the threshold value, the image decoding device 200 increments the binarization parameter (cParam) (S253 and S254 in Fig. 18). Then, if cParam from the immediately preceding block of coefficients is greater than 0, the image decoding device 200 changes the context (in other words, increments the number of context sets) (S184 and S185 in Fig. 10).

[0196] In other words, based on a variable generated in arithmetic decoding over the immediately preceding block of coefficients, the image decoding device 200 determines whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value.

[0197] More specifically, if the absolute value of the target coefficient is greater than 2, the image decoding device 200 converts the remainder, which is obtained by subtracting 3 from the absolute value, into a multi-valued value in accordance with a lookup table designated by a binarization parameter, among a plurality of lookup tables. This binarization parameter is determined depending on whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. The above-mentioned variable is this binarization parameter.

[0198] As above, the threshold value may be a predetermined value ("1"). In this case, as in the previous case, the image decoding device 200 may determine whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than 1, based on a variable generated in arithmetic coding over the immediately preceding block of coefficients. For example, this variable is greater1_flag. In other words, the image decoding device 200 may change the context in accordance with whether the immediately preceding block of coefficients includes a coefficient that has greater1_flag having a value of 1.

[0199] (Implementation variant 3)

In Embodiment 3, a variation of Embodiment 1 is described. The image encoding device according to Embodiment 3 differs from the image encoding device according to Embodiment 1 in that the layer encoding unit 133A functions differently from the layer encoding unit 133. The layer encoding unit 133A is described below.

[0200] <Structure of Level Coding Module 133A>

Fig. 34 is a block diagram showing the structure of the layer encoding unit 133A. The layer encoding unit 133A includes not only the structure of the layer encoding unit 133 according to Embodiment 1, but also the greater_flag switching unit 160. In addition, the functions of the residual setting unit 143A and the context set selecting unit 145A are different from the functions of the residual setting unit 143 and the context set selecting unit 145.

[0201] <Processing (Level Encoding)>

Next, with reference to Fig. 35, the level encoding will be described in more detail. The level encoding according to Embodiment 1 is supplemented with new steps S501 to S503. In addition, steps S141 and S171 are different from steps S141 and S171. The steps identical to those in Embodiment 1 are not explained again below.

[0202] (Step S141)

The context set selection module 145A sets the number of context sets to be used in arithmetic coding over greater1_flag and greater2_flag. This step will be described in more detail below.

[0203] (Step S501)

The greater_flag switching unit 160 determines whether the binarization parameter (cParam) that was updated in the processing over the immediately preceding coefficient block is greater than 0. In the same manner as described in embodiment 1, if the binarization parameter is 0, then greater1_flag and greater2_flag are encoded. On the other hand, if the binarization parameter is greater than 0, then greater1_flag and greater2_flag are not encoded.

[0204] (Steps S502, S503 and S171)

If the binarization parameter (cParam) that has been updated for the immediately preceding block of coefficients is greater than 0, the residual setting unit 143A sets the base value of the residual to 1. If the binarization parameter (cParam) is 0, the residual setting unit 143A sets the base value of the residual to 3. Then, if the absolute value of the target coefficient is greater than or equal to the base value of the residual, the layer encoding unit 133A encodes the residual in steps S172 to S175. On the other hand, if the absolute value is less than the base value of the residual, the layer encoding unit 133A does not encode the residual. If the residual is encoded, the layer encoding unit 133A subtracts the base value of the residual from the absolute value and sets the resulting value as the residual and the encoded one. This is because greater1_flag and greater2_flag are sometimes not encoded depending on cParam, and the value to be set as the residual therefore changes. In embodiment 1, since greater1_flag and greater2_flag are present, whatever cParam is, the base value of the remainder is fixed to "3".

[0205] <Processing (Selecting Context Sets)>

Next, with reference to Fig. 35, the selection of context sets (S141 in Fig. 34) is described in more detail.

[0206] (Steps S511 to S513)

The context set selection unit 145A determines whether the target block of coefficients has the lowest frequency in the target coding block. If the target block of coefficients has the lowest frequency, the context set selection unit 145A sets the number of context sets to 0. Otherwise, the context set selection unit 145A sets the number of context sets to 1. In other words, if the target block of coefficients is located in the upper left corner of the target coding block, the context set selection unit 145A sets the number of context sets to 0. Otherwise, the context set selection unit 145A sets the number of context sets to 1.

[0207] Unlike embodiment 1, context set selection module 145A does not change the context set when cParam is greater than 0. This is because greater1_flag and greater2_flag are not coded and the context is not selected if cParam is greater than 0, as shown in the previously described layer encoding flow.

[0208] <Effects>

As described above, the image encoding device according to the present embodiment determines based on the binarization parameter that has been updated for the immediately preceding block of coefficients that greater1_flag and greater2_flag do not need to be encoded, and therefore the coefficient is encoded based on the residual instead of greater1_flag and greater2_flag. In this way, if there is at least one coefficient greater than the threshold in the processing for the immediately preceding block of coefficients, the image encoding device performs bypass arithmetic encoding that does not use any context, instead of arithmetic encoding that uses context. Bypass arithmetic encoding does not require loading and updating a context. In addition, bypass arithmetic encoding makes it possible to start processing without waiting for the context to be updated in a preliminary stage. Therefore, bypass arithmetic encoding can increase the processing speed more than arithmetic encoding that uses context.

[0209] In addition, if there is a high probability that the absolute value of the target coefficient is small, greater1_flag and greater2_flag are sometimes equal to 0, so that encoding the residual is not necessary. However, this situation rarely occurs when there is a high probability that the absolute value of the target coefficient is large. Thus, the total bin value can often be reduced if greater1_flag and greater2_flag are not encoded.

[0210] If cParam after processing on the immediately preceding block of coefficients is greater than 0 (in other words, if there is at least one coefficient greater than the threshold value), the image encoding device according to the present embodiment expects that there is a high probability of obtaining coefficients having an absolute value in the target block of coefficients, and therefore does not encode greater1_flag and greater2_flag. As a result, the processing can be performed at a higher speed by reducing the arithmetic encoding steps using contexts while suppressing the deterioration in encoding efficiency.

[0211] In addition, greater1_flag and greater2_flag are not coded when cParam is greater than 0. As a result, a set of contexts that is used when cParam is greater than 0, in other words, a set of contexts that is used when there is a high probability of providing coefficients having a large value, is not necessary. More specifically, if the target coefficient block is located in the upper left corner of the target encoding block, the image encoding device uses one context. If the target coefficient block is not in the upper left corner, the image encoding device uses one common set of contexts. As a result, it is possible to reduce the number of contexts used and thereby reduce the size of the memory for storing contexts and the size of the circuit for selecting contexts.

[0212] By combining information to be used in determining whether to encode greater1_flag and greater2_flag and information to be used in encoding the residual, the size of the circuit is reduced. More specifically, cParam, which is incremented each time the absolute value of the target coefficient exceeds a threshold, is used to change the binarization parameter of the residual and to determine whether to encode greater1_flag and greater2_flag. As a result, one structure can perform various functions. Compared with the existing HEVC standard (Non-Patent Document 1), the image encoding device according to the present embodiment can determine whether to encode greater1_flag and greater2_flag without including an additional structure.

[0213] Furthermore, the image encoding method according to the present embodiment was applied to test software according to the present HEVC standard (Non-Patent Document 1) for experiments. FIG. 37 shows the experimental results compared with the results without applying the present embodiment. The experimental conditions are based on the general experimental conditions of the HEVC standard group. The numerical values in FIG. 37 are the results with respect to the first 49 frames of the test image. A larger numerical value indicates lower encoding efficiency. A negative value indicates an improvement in encoding efficiency. As shown in FIG. 37, each of the values is in the range of -0.00% to 0.06%. This means that the encoding efficiency is hardly affected even if the processing speed is increased by eliminating arithmetic encoding that uses contexts, or even if the total number of contexts is reduced.

[0214] It should be noted that it has been described above that the image encoding device determines whether to encode greater1_flag and greater2_flag based on the binarization parameter (cParam) that has been updated for the immediately preceding coefficient block. However, the determination may be made based on the binarization parameter that has been updated for a coefficient block other than the immediately preceding coefficient block. For example, the image encoding device may determine that greater1_flag and greater2_flag should not be encoded for the following coefficient blocks if cParam exceeds 0 in the processing for any processed coefficient block. The coefficient blocks are processed sequentially from the higher frequency block. Thus, if cParam of any of the coefficient blocks exceeds 0, in other words, if a coefficient having an absolute value that is greater than or equal to the threshold value is found, then there is a high probability that all subsequent coefficient blocks have greater coefficient values. In the above case, even if greater1_flag and greater2_flag are not encoded, the encoding efficiency is sometimes not reduced.

[0215] It should also be noted that it has been described above that the image encoding device does not encode greater1_flag and greater2_flag if cParam is greater than 0, but the present embodiment is not limited to the above. For example, it is also possible to not encode greater1_flag and greater2_flag if cParam is greater than 1 or greater than 2. Depending on the input image, encoding efficiency can sometimes be improved if the determination of whether to encode greater1_flag and greater2_flag is made based on whether cParam is greater than 1.

[0216] The rest is the same as described in Embodiment 1, so no further explanation is given here.

[0217] (Embodiment 4)

In Embodiment 4, a description is given of an image decoding device that decodes an encoded bit stream generated by the image encoding device according to Embodiment 3. The image decoding device according to Embodiment 4 differs from the image decoding device according to Embodiment 2 in that the layer decoding unit 232 is replaced with the layer decoding unit 232A. The layer decoding unit 232A is described below.

[0218] <Structure of Level Decoding Module 232A>

Fig. 38 is a block diagram showing the structure of the layer decoding unit 232A. The layer decoding unit 232A includes not only the structure of the layer decoding unit 232 according to Embodiment 2, but also the greater_flag switching unit 260. In addition, the functions of the context set selection unit 241A and the arithmetic decoding unit 247A differ from the corresponding functions of the context set selection unit 241 and the arithmetic decoding unit 247.

[0219] <Processing (Level Decoding)>

Next, with reference to Fig. 39, the level decoding will be described in more detail. The level decoding according to Embodiment 2 is supplemented with new steps S601 to S603. In addition, steps S341A and S371A are different from steps S341 and S371, respectively. It should be noted that the steps identical to those in Embodiment 2 are not explained again hereinafter.

[0220] (Step S341A)

The context set selection unit 241A sets (selects) the respective numbers of context sets to be used in the arithmetic decoding over greater1_flag and the arithmetic decoding over greater2_flag. The selection method is the same as in the context set selection (FIG. 36) according to Embodiment 3. In other words, the context set selection unit 241A does not select the context based on cParam.

[0221] (Step S601)

The greater_flag switching module 260 260 determines whether the binarization parameter (cParam) that was updated in the processing over the immediately preceding block of coefficients is greater than 0. In the same manner as described in embodiment 2, the greater_flag switching module 260 260 decodes greater1_flag and greater2_flag if the binarization parameter is 0, but does not decode greater1_flag and greater2_flag if the binarization parameter is greater than 0.

[0222] (Steps S602, S603 and S371A)

If the binarization parameter (cParam) that has been updated for the immediately preceding block of coefficients is greater than 0, then the arithmetic decoding unit 247A sets the base value of the residual to 1. If the binarization parameter (cParam) is 0, then the residual setting unit 247A sets the base value of the residual to 3. If the base value of the residual is 3 and greater2_flag of the target coefficient is 1, or if the base value of the residual is 1 and significant_flag of the target coefficient is 1, then the layer decoding unit 232A decodes the residual in steps S372 and S373. Otherwise, the layer decoding unit 232A does not decode the residual. If the residual is decoded, the numerical value obtained by adding the base value of the residual to the decoded residual is the absolute value of the target coefficient.

[0223] <Effects>

As described above, the image decoding device according to the present embodiment can provide the same effects as described in Embodiment 3.

[0224] In each of the above-described embodiments, each of the functional blocks may be implemented, as a rule, as an MPU (microprocessor), a memory, and the like. In addition, the processing performed by each of the functional blocks may be implemented, as a rule, by software (program), and the software is recorded on a recording medium, such as a ROM. Then, such software may be distributed, for example, by downloading, or may be recorded on a recording medium, such as a ROM on a compact disc (CD-ROM), for subsequent distribution. In addition, each of the functional blocks may, of course, be implemented in hardware (dedicated circuit).

[0225] The processing described in each of the embodiments may be performed as centralized processing by using one device (system) or as decentralized processing by using a plurality of devices. In addition, the program described above may be executed by one computer or a plurality of computers. In other words, both centralized processing and decentralized processing may be performed by the program.

[0226] Although the image encoding device and the image decoding device have been described with reference to multiple embodiments as described above, the present invention is not limited to these embodiments.

[0227] It should also be noted that the processing modules in each of the image encoding device and the image decoding device according to the above-described embodiments are typically implemented on a highly integrated circuit (LSI). They may be integrated individually, or some or all of them may be combined in a single chip.

[0228] It should also be noted that the integrated circuit method is not limited to the LSI, and it can be implemented as a specialized circuit or a general-purpose processor. In addition, it is possible to use a field-programmable gate array (FPGA), which can be programmed after the LSI is manufactured, or a reconfigurable processor in which the connection and configuration of circuit cells within the LSI can be reconfigured.

[0229] Each of the structural elements in each of the above-described embodiments may be configured in the form of a specialized hardware product or may be realized by executing software suitable for the structural element. Each of the structural elements may be realized by a program executing module, such as a CPU and a processor, reading and executing a software program recorded on a recording medium, such as a hard disk or a semiconductor memory.

[0230] In other words, each of the image encoding device and the image decoding device includes: a control circuit and a storage means electrically connected to the control circuit and accessible from the control circuit. The control circuit includes at least one of specialized hardware and a program executing module. The storage means stores a software program executed by the program executing module when the control circuit includes the program executing module.

[0231] In addition, the present invention may be the above-described program, or may be a non-transitory computer-readable recording medium on which the program is recorded. Of course, the program may be distributed via a transmission medium such as the Internet.

[0232] Furthermore, all numerical expressions in the above description are examples for explaining the present invention in more detail. The present invention is not limited to such numerical examples.

[0233] In addition, the division of functional blocks in the block diagram is given as examples. It is also possible that a plurality of functional blocks are implemented as one functional block, that one functional block is divided into a plurality of parts, or that a partial function is moved to another functional block. In addition, the functions of a plurality of functional blocks having similar functions can be performed in parallel or in a time division mode by a common single hardware or software.

[0234] It should also be noted that the order of execution of the steps included in each of the above-described image encoding method and the above-described image decoding method is an example for explaining the present invention in more detail. Thus, various orders other than the above-described order may be used. Some of the steps may be executed simultaneously (in parallel) with another step.

[0235] Thus, although only some exemplary embodiments of the image encoding device and the image decoding device according to the present invention have been described in detail above, the present invention is not limited to these embodiments. It will be easily understood by those skilled in the art that various modifications of the exemplary embodiments and combinations of structural elements of the various embodiments are possible without substantially departing from the solutions and advantages of the present invention. Accordingly, all such modifications and combinations are intended to be included within the scope of the present invention.

[0236] (Embodiment 5)

The processing described in each of the embodiments can be simply implemented in an independent computer system by recording, on a recording medium, a program for implementing the configurations of the moving image encoding method (image encoding method) and the moving image decoding method (image decoding method) described in each of the embodiments of the invention. The recording medium may be any recording medium as long as the program can be recorded, such as a magnetic disk, an optical disk, a magneto-optical disk, an IC card, and a semiconductor memory.

[0237] Next, applications of the moving image encoding method (image encoding method) and the moving image decoding method (image decoding method) described in each of the embodiments and a system using them will be described. The system has a feature of having an image encoding and decoding device that includes an image encoding device using the image encoding method and an image decoding device using the image decoding method. Other configurations in the system can be changed as necessary depending on cases.

[0238] Fig. 40 illustrates a general configuration of a content providing system ex100 for implementing content distribution services. A communication service providing area is divided into cells of a desired size, and base stations ex106, ex107, ex108, ex109 and ex110, which are fixed wireless stations, are located in each of the cells.

[0239] The content providing system ex100 is connected to devices such as a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a cellular telephone ex114, and a gaming machine ex115, via the Internet ex101, an Internet service provider ex102, a telephone network ex104, and base stations ex106 to ex110, respectively.

[0240] However, the configuration of the content providing system ex100 is not limited to the configuration shown in Fig. 40, and a combination in which any of the elements are connected is acceptable. In addition, each device can be directly connected to the telephone network ex104, and not through the base stations ex106 to ex110, which are fixed wireless stations. In addition, these devices can be connected to each other via short-range wireless communication, etc.

[0241] The camera ex113, for example, a digital video camera, is capable of capturing video. The camera ex116, for example, a digital camera, is capable of capturing both still images and video. In addition, the cellular telephone ex114 may be one that satisfies any of the standards such as Global System for Mobile Communications (GSM) (registered trademark), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), and High Speed Packet Access (HSPA). In addition, the cellular telephone ex114 may be a personal mobile phone system (PHS).

[0242] In the content providing system ex100, the streaming server ex103 is connected to the camera ex113 and other elements via the telephone network ex104 and the base station ex109, which enables distribution of live images and the like. In such distribution, the content (for example, a video of a live music show) captured by the user via the camera ex113 is encoded as described above in each of the embodiments (i.e., the camera functions as an image encoding device according to an aspect of the present invention), and the encoded content is transmitted to the streaming server ex103. On the other hand, the streaming server ex103 distributes the stream of transmitted content data to clients upon their requests. The clients include a computer ex111, a PDA ex112, a camera ex113, a cellular phone ex114 and a gaming machine ex115, which are capable of decoding the above-mentioned encoded data. Each of the devices that has received the distributed data decodes the data and reproduces the encoded data (i.e., functions as an image decoding device in accordance with one aspect of the present invention).

[0243] The captured data may be encoded by the camera ex113 or the streaming server ex103 that transmits the data, or the encoding processes may be divided between the camera ex113 and the streaming server ex103. In addition, the distributed data may be decoded by the clients or the streaming server ex103, or the decoding processes may be divided between the clients and the streaming server ex103. In addition, still image and video data captured not only by the camera ex113 but also by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The encoding processes may be performed by the camera ex116, the computer ex111, or the streaming server ex103 or divided between them.

[0244] In addition, the encoding and decoding processes can be performed by an LSI ex500 typically included in each of the computer ex111 and the devices. The LSI ex500 can be configured from a single chip or a plurality of chips. Software for encoding and decoding a video can be integrated into some type of recording medium (for example, a CD-ROM, a floppy disk, and a hard disk) that can be read by the computer ex111 and the like, and the encoding and decoding processes can be performed using the software. In addition, when the cellular telephone ex114 is equipped with a camera, video data obtained by the camera can be transmitted. The video data is data encoded by the LSI ex500 included in the cellular telephone ex114.

[0245] In addition, the streaming server ex103 may be composed of servers and computers and may perform data decentralization and process decentralized data, record or distribute data.

[0246] As described above, clients can receive and reproduce encoded data in the content providing system ex100. In other words, clients can receive and decode information transmitted by a user and reproduce the decoded data in real time in the content providing system ex100, so that a user who does not have any specific right and equipment can realize personal broadcasting.

[0247] In addition to the example of the content providing system ex100, at least one of the moving image encoding device (image encoding device) and the moving image decoding device (image decoding device) described in each of the embodiments can be implemented in the digital broadcasting system ex200 shown in Fig. 41. More specifically, the broadcasting station ex201 communicates or transmits, by radio waves, to the broadcasting satellite ex202, multiplexed data obtained by multiplexing audio data and the like onto video data. The video data is data encoded by the moving image encoding method described in each of the embodiments (i.e., data encoded by the image encoding device according to one aspect of the present invention). After receiving the multiplexed data, the broadcasting satellite ex202 transmits radio waves to perform broadcasting. Then, the home-use antenna ex204 with a satellite broadcast reception function receives radio waves. Then, a device such as a television (receiver) ex300 and a set-top box (STB) ex217 decodes the received multiplexed data and reproduces the decoded data (i.e., functions as an image decoding device according to one aspect of the present invention).

[0248] In addition, the reading/writing device ex218(i) reads and decodes multiplexed data recorded on the recording medium ex215 such as a DVD and BD, or (i) encodes video signals on the recording medium ex215, and in some cases, records data obtained by multiplexing an audio signal onto the encoded data. The reading/writing device ex218 may include a moving image decoding device or a moving image encoding device, as shown in each of the embodiments of the invention. In this case, the reproduced video signals are displayed on the monitor ex219 and can be reproduced by another device or system using the recording medium ex215 on which the multiplexed data is recorded. In addition, it is possible to implement a moving image decoding device in a set-top box device ex217 connected to a cable ex203 for cable television, or to an antenna ex204 for satellite and/or terrestrial broadcasting, in order to display video signals on a monitor ex219 of a television set ex300. The moving image decoding device may be implemented not in a set-top box, but in a television set ex300.

[0249] Fig. 42 illustrates a television (receiver) ex300 that uses the moving picture encoding method and the moving picture decoding method described in each of the embodiments. The television ex300 includes: a tuner ex301 that receives and provides multiplexed data obtained by multiplexing audio data on video data, via an antenna ex204 or a cable ex203, etc. that receives broadcasting; a modulation/demodulation unit ex302 that demodulates the received multiplexed data or modulates the data into multiplexed data to be provided to the outside; and a multiplexing/demultiplexing unit ex303 that demultiplexes the modulated multiplexed data into video data and audio data or multiplexes the video data and audio data encoded by the signal processing unit ex306 into data.

[0250] The television ex300 further includes: a signal processing unit ex306 including an audio signal processing unit ex304 and a video signal processing unit ex305 that decode audio data and video data and encode the audio data and video data, respectively (which function as an image encoding device and an image decoding device according to aspects of the present invention); and an output unit ex309 including a speaker ex307 that provides a decoded audio signal, and a display unit ex308 that displays the decoded video signal, such as a display. In addition, the television ex300 includes an interface unit ex317 including an operation input unit ex312 that receives an input of a user operation. In addition, the television ex300 includes a control module ex310 that controls each of the constituent elements of the television ex300 as a whole, and a power supply circuit module ex311 that supplies power to each of the elements. In addition to the operation input module ex312, the interface module ex317 may include: a bridge ex313 that is connected to an external device such as a reader/writer module ex218; a socket module ex314 for providing a mounting of a recording medium such as an SD card; a driver ex315 to be connected to an external recording medium such as a hard disk; and a modem ex316 to be connected to a telephone network. Here, the recording medium ex216 can electrically record information using a non-volatile/volatile memory element for storing. The constituent elements of the television ex300 are connected to each other via a synchronous bus.

[0251] First, a configuration will be described in which a television ex300 decodes multiplexed data received from the outside via an antenna ex204 or the like and reproduces the decoded data. In the television ex300, upon a user operation via a remote controller ex220 or the like, a multiplexing/demultiplexing unit ex303 demultiplexes the multiplexed data demodulated by a modulation/demodulation unit ex302, under the control of a control unit ex310 including a CPU. Furthermore, an audio signal processing unit ex304 decodes the demultiplexed audio data, and a video signal processing unit ex305 decodes the demultiplexed video data, using the decoding method described in each of the embodiments, in the television ex300. The output module ex309 outputs the decoded video signal and the audio signal to the outside, respectively. When the output module ex309 outputs the video signal and the audio signal, the signals may be temporarily stored in the buffers ex318 and ex319 and others, so that the signals are reproduced synchronously with each other. In addition, the television ex300 can read the multiplexed data not through broadcasting etc., but from the recording medium ex215 and ex216, such as a magnetic disk, an optical disk, and an SD card. Next, a configuration will be described in which the television ex300 encodes the audio signal and the video signal and transmits the data to the outside or records the data on the recording medium. In the television ex300, upon an operation by a user via a remote controller ex220 or the like, the audio signal processing unit ex304 codes an audio signal, and the video signal processing unit ex305 codes a video signal, under the control of the control unit ex310, using the encoding method described in each of the embodiments. The multiplexing/demultiplexing unit ex303 multiplexes the coded video signal and the audio signal and provides the resulting signal to the outside. When the multiplexing/demultiplexing unit ex303 multiplexes the video signal and the audio signal, the signals may be temporarily stored in the buffers ex320 and ex321 and others, so that the signals are reproduced in synchronism with each other. Here, the buffers ex318, ex319, ex320 and ex321 may be in a plurality as shown, or at least one buffer may be shared in the television ex300. In addition, data can be stored in a buffer so that overflow or underflow of the system can be avoided, for example, between the ex302 modulation/demodulation module and the ex303 multiplexing/demultiplexing module.

[0252] In addition, the television ex300 may include a configuration for receiving an AV input from a microphone or a camera, different from a configuration for receiving audio and video data from a broadcast or a recording medium, and may encode the received data. Although the television ex300 may encode, multiplex, and provide external data as described, it may only be capable of receiving, decoding, and providing external data, but not encoding, multiplexing, and providing external data.

[0253] In addition, when the reading/writing module ex218 reads or writes multiplexed data from or to the recording medium, one of the television ex300 and the reading/writing module ex218 may decode or encode the multiplexed data, and the television ex300 and the reading/writing module ex218 may share decoding or encoding.

[0254] As an example, Fig. 43 illustrates a configuration of an information reproducing/recording unit ex400 when data is read from or written to an optical disc. The information reproducing/recording unit ex400 includes constituent elements ex401, ex402, ex403, ex404, ex405, ex406 and ex407, which will be described below. The optical head ex401 irradiates a laser spot on a recording surface of a recording medium ex215, which is an optical disc for recording information, and detects reflected light from the recording surface of the recording medium ex215 to read information. The modulation recording unit ex402 electrically drives a semiconductor laser included in the optical head ex401 and modulates the laser light in accordance with the recorded data. The playback demodulation module ex403 amplifies the playback signal obtained by electrically detecting reflected light from the recording surface using a photodetector included in the optical head ex401, and demodulates the playback signal by separating a component of the signal recorded on the recording medium ex215 to reproduce the necessary information. The buffer ex404 temporarily stores information to be recorded on the recording medium ex215 and information reproduced from the recording medium ex215. The disc motor ex405 rotates the recording medium ex215. The servo control module ex406 moves the optical head ex401 to a predetermined information track, while controlling the rotation drive of the disc motor ex405 so as to follow the laser spot. The system control module ex407 carries out overall control of the information playback/recording module ex400. The reading and writing processes can be realized by a system control module ex407 using various information stored in a buffer ex404 and generating and adding new information as needed, and by a modulation recording module ex402, a playback demodulation module ex403, and a servo control module ex406 that record and reproduce information through an optical head ex401 during operation in a coordinated manner. The system control module ex407 includes, for example, a microprocessor, and executes processing by causing a computer to execute a program for reading and writing.

[0255] Although the ex401 optical head irradiates a laser spot as described, it can perform high-density recording using near-field light.

[0256] Fig. 44 illustrates a recording medium ex215 that is an optical disc. On the recording surface of the recording medium ex215, guide grooves are spirally formed, and an information track ex230 records in advance address information indicating an absolute position on the disc according to a change in the shape of the guide grooves. The address information includes information for specifying positions of recording blocks ex231 that are a unit for recording data. Reproducing the information track ex230 and reading the address information in an apparatus that records and reproduces data can result in specifying the positions of the recording blocks. Furthermore, the recording medium ex215 includes a data recording area ex233, an inner peripheral area ex232 and an outer peripheral area ex234. The data recording area ex233 is an area for use in recording user data. An inner peripheral area ex232 and an outer peripheral area ex234, which are located inside and outside the data recording area ex233, respectively, are intended for special use except for recording user data. The information reproducing/recording unit 400 reads and writes coded audio data, coded video data, or multiplexed data obtained by multiplexing coded audio and video data from and to the data recording area ex233 of the recording medium ex215.

[0257] Although an optical disc having a layer such as a DVD and a BD is described as an example in the description, the optical disc is not limited to this, and may be an optical disc having a multilayer structure and capable of being recorded on a part rather than the entire surface. In addition, the optical disc may have a structure for multidimensional recording/reproduction, such as recording information using light of colors with different wavelengths in the same part of the optical disc, and for recording information using different layers at different angles.

[0258] In addition, the automobile ex+210 having the antenna ex205 can receive data from the satellite ex202 and others, and reproduce video on a display device such as the automobile navigation system ex+211 installed in the automobile ex210, in the digital broadcasting system ex200. Here, the configuration of the automobile navigation system ex211 will be a configuration including, for example, a GPS receiving module of the configuration shown in Fig. 42. The same will be true for the configuration of the computer ex111, the cellular phone ex114, etc.

[0259] Fig. 45A illustrates a cellular phone ex114 that uses the moving image encoding method and the moving image decoding method described in the embodiments. The cellular phone ex114 includes: an antenna ex350 for transmitting and receiving radio waves via a base station ex110; a camera module ex365 capable of capturing a moving and a still image; and a display module ex358 such as a liquid crystal display for displaying data such as decoded video captured by the camera module ex365 or received by the antenna ex350. The cellular phone ex114 further includes: a main body including an operation key module ex366; an audio output module ex357 such as a speaker for outputting audio; an audio input module ex356 such as a microphone for inputting audio; a memory module ex367 for storing captured video or still images, recorded audio, encoded or decoded data of received video, still images, email or other; and a socket module ex364, which is an interface module for a recording medium that stores data in the same manner as the memory module ex367.

[0260] Next, an example of a configuration of a cellular phone ex114 will be described with reference to Fig. 45B. In the cellular phone ex114, a main control module ex360 for generally controlling each module of the main body including the display module ex358 and the operation key module ex366 is mutually connected via a synchronous bus module ex370 to a power supply circuit module ex361, an operation input control module ex362, a video signal processing module ex355, a camera interface module ex363, a liquid crystal display (LCD) control module ex359, a modulation/demodulation module ex359, a multiplexing/demultiplexing module ex353, an audio signal processing module ex354, a socket module ex364, and a memory module ex367.

[0261] When the end call key or the power key is turned on by a user operation, the power control module ex361 supplies the corresponding modules with power from the battery to activate the cellular telephone ex114.

[0262] In the cellular phone ex114, the audio signal processing module ex354 of the unit ex354 converts audio signals received by the audio input module ex356 in the voice dialogue mode into digital audio signals under the control of a main control module ex360 including a CPU, a ROM and a RAM. Then, the modulation/demodulation module ex352 performs spread spectrum processing on the digital audio signals, and the transmission and reception module ex351 performs digital-to-analog conversion and frequency conversion on the data so as to transmit the resulting data via the antenna ex350. Further, in the cellular phone ex114, the transmission and reception module ex351 amplifies the data received by the antenna ex350 in the voice dialogue mode and performs frequency conversion and analog-to-digital conversion on the data. Then, the modulation/demodulation module ex352 performs spread spectrum inversion processing on the data, and the audio signal processing module ex354 converts it into analog audio signals to output them through the audio output module ex357.

[0263] Furthermore, when an e-mail is transmitted in the data transmission mode, text data of the e-mail inputted by operating the operation key module ex366 etc. of the main body is sent to the main control module ex360 via the operation input control module ex362. The main control module ex360 causes the modulation/demodulation module ex352 to perform spread spectrum processing on the text data, and the transmission and reception module ex351 performs digital-to-analog conversion and frequency conversion on the resultant data to transmit the data to the base station ex110 via the antenna ex350. When the e-mail is received, processing that is approximately the inverse of the processing for transmitting the e-mail is performed on the received data, and the resultant data is provided to the display module ex358.

[0264] When video, still images, or video and audio are transmitted in a data transmission mode, the video signal processing unit ex355 compresses and codes video signals supplied from the camera unit ex365 using the moving image encoding method shown in each of the embodiments (i.e., functions as an image encoding device according to an aspect of the present invention), and transmits the encoded video data to the multiplexing/demultiplexing unit ex353. In contrast, when the camera unit ex365 captures video, still images, etc., the audio signal processing unit ex354 codes audio signals received by the audio input unit ex356 and transmits the encoded audio data to the multiplexing/demultiplexing unit ex353.

[0265] The multiplexing/demultiplexing unit ex353 multiplexes the encoded video data supplied from the video signal processing unit ex355 and the encoded audio data supplied from the audio signal processing unit ex354 using a predetermined method. Then, the modulation/demodulation unit ex352 (modulation/demodulation circuit unit) performs spread spectrum processing on the multiplexed data, and the transmission and reception unit ex351 performs digital-to-analog conversion and frequency conversion on the data to transmit the resulting data via the antenna ex350.

[0266] When receiving data from a video file that is linked to a web page, etc. in a data transfer mode, or when receiving an e-mail message with a video and/or audio attachment, in order to decode the multiplexed data received via the antenna ex350, the multiplexing/demultiplexing unit ex353 demultiplexes the multiplexed data into a video data bitstream and an audio data bitstream and supplies the encoded video data to the video signal processing unit ex355 and the encoded audio data to the audio signal processing unit ex354 via the synchronous bus ex370. The video signal processing unit ex355 decodes the video signal using the moving image decoding method corresponding to the moving image encoding method shown in each of the embodiments (i.e., functions as an image decoding device according to an aspect of the present invention), and then the display unit ex358 displays, for example, the video and still images included in the video file linked to the web page, via the LCD control unit ex359. In addition, the audio signal processing unit ex354 decodes the audio signal, and the audio output unit ex3578 outputs the audio.

[0267] Furthermore, like the television ex300, a terminal such as the cellular telephone ex114 is likely to have 3 types of implementation configurations including not only (i) a transmitting and receiving terminal including both an encoding device and a decoding device, but also (ii) a transmitting terminal including only an encoding device, and (iii) a receiving terminal including only a decoding device. Although the digital broadcasting system ex200 in the description receives and transmits multiplexed data obtained by multiplexing audio data onto video data, the multiplexed data may be data obtained by multiplexing not audio data but character data related to video onto video data, and may not be multiplexed data but video data itself.

[0268] Thus, the method for coding moving images and the method for decoding a moving image in each of the embodiments can be used in any of the described devices and systems. Thus, the advantages described in each of the embodiments can be obtained.

[0269] Furthermore, various modifications and changes can be made in any of the embodiments in the present invention.

[0270] (Embodiment 6)

The video data may be generated by switching, as necessary, between (i) the moving picture coding method or the moving picture coding device shown in each of the embodiments and (ii) the moving picture coding method or the moving picture coding device in accordance with another standard such as MPEG-2, MPEG-4 AVC and VC-1.

[0271] Here, when a plurality of video data that conforms to different standards is generated and then decoded, decoding methods must be selected to conform to the different standards. However, since it is impossible to determine the standard that each of the plurality of video data to be decoded conforms to, there is a problem that an appropriate decoding method cannot be selected.

[0272] In order to solve this problem, multiplexed data obtained by multiplexing audio data, etc., onto video data has a structure including identification information indicating which standard the video data conforms to. A specific structure of multiplexed data including video data generated in the moving image encoding method and the moving image encoding device shown in each of the embodiments will be described below. The multiplexed data is a digital stream in the MPEG-2 transport stream format.

[0273] Fig. 46 illustrates the structure of multiplexed data. As shown in Fig. 46, the multiplexed data can be obtained by multiplexing at least one of a video stream, an audio stream, a presentation graphics (PG) stream, and an interactive graphics stream. The video stream represents a primary video and a secondary video of a movie, the audio stream (IG) represents a primary audio part and a secondary audio part that is to be mixed with the primary audio part, and the presentation graphics stream represents subtitles of the movie. Here, the primary video is a normal video to be displayed on the screen, and the secondary video is a video to be displayed in a smaller window in the primary video. In addition, the interactive graphics stream represents an interactive screen that will be generated by placing graphical user interface (GUI) components on the screen. The video stream is encoded in the moving picture encoding method or the moving picture encoding device shown in each of the embodiments, or in the moving picture encoding method or the moving picture encoding device in accordance with a conventional standard such as MPEG-2, MPEG-4 AVC and VC-1. The audio stream is encoded in accordance with standards such as Dolby-AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD and linear PCM.

[0274] Each stream included in the data multiplex is identified by a PID. For example, 0x1011 is allocated to a video stream to be used for the movie video, 0x1100 to 0x111F are allocated to an audio stream, 0x1200 to 0x121F are allocated for presentation graphics streams, 0x1400 to 0x141F are allocated for interactive graphics streams, 0x1B00 to 0x1B1F are allocated for video streams to be used for the secondary movie video, and 0x1A00 to 0x1A1F are allocated for audio streams to be used for secondary audio to be mixed with the primary audio.

[0275] Fig. 47 schematically illustrates how the data is multiplexed. First, a video stream ex235 consisting of video frames and an audio stream ex238 consisting of audio frames are converted into a stream of PES packets ex236 and a stream of PES packets ex239 and then into TS packets ex237 and TS packets ex240, respectively. In addition, data of a presentation graphics stream ex241 and data of an interactive graphics stream ex244 are converted into a stream of PES packets ex242 and a stream of PES packets ex245 and then into TS packets ex243 and TS packets ex246, respectively. These TS packets are multiplexed into a stream to obtain multiplexed data ex247.

[0276] Fig. 48 shows in more detail how a video stream is stored in a PES packet stream. The first row of Fig. 48 shows a stream of video frames in the video stream. The second row shows a stream of PES packets. As shown by the arrows labeled yy1, yy2, yy3 and yy4 in Fig. 48, the video stream is divided into pictures (images) as I-pictures, B-pictures and P-pictures, each of which is a unit of video presentation, and the pictures are stored in the payload of each of the PES streams. Each of the PES packets has a PES header, and the PES header stores a presentation timestamp (PTS) indicating the display time of the picture and a decoding timestamp (DTS) indicating the decoding time of the picture.

[0277] Fig. 49 illustrates the format of TS packets to be finally written onto the multiplexed data. Each of the TS packets is a 188-byte fixed-length packet including a 4-byte TS header having information such as a PID for identifying a stream, and a 184-byte TS payload for storing data. PES packets are divided and stored in the TS payloads, respectively. When using a BD ROM, each of the TS packets is given a 4-byte TP_Extra_Header (TP extra header), which results in 192-byte source packets. The source packets are written onto the multiplexed data. TP_Extra_Header stores information such as Arrival_Time_Stamp (ATS). ATS indicates the start transmission time at which each of the TS packets should be transmitted to the PID filter. The source packets are ordered into multiplexed data as shown at the bottom of Fig. 49. The numbers increasing from the start of the multiplexed data are called source packet numbers (SPNs).

[0278] Each of the TS packets included in the multiplexed data includes not only audio, video, subtitle and other streams, but also a program table with a list of stream programs with their identifiers (PAT), a program structure table (PMT) and a reference clock field (PCR). The PAT indicates which PID in the PMT is used in the multiplexed data, and the PID of the PAT itself is registered as zero. The PMT stores the PIDs of the video, audio, subtitle and other streams included in the multiplexed data, and the attribute information of the streams corresponding to the PIDs. The PMT also has various descriptors related to the multiplexed data. The descriptors have information such as copy control information indicating whether copying of the multiplexed data is permitted or not. The PCR stores STC time information corresponding to the ATS indicating when the PCR packet is transmitted to the decoder in order to achieve synchronization between the arrival time clock (ATC), which is the ATS time axis, and the system time clock (STC), which is the PTS and DTS time axis.

[0279] Fig. 50 illustrates the structure of the PMT data in more detail. A PMT header is located at the top of the PMT. The PMT header describes the length of the data included in the PMT, etc. A plurality of descriptors related to the multiplexed data are located after the PMT header. Information such as copy control information is described in the descriptors. Following the descriptors, a plurality of pieces of stream information related to the streams included in the multiplexed data are located. Each piece of stream information includes stream descriptors, each of which describes information such as a stream type for identifying a stream compression codec, a stream PID, and stream attribute information (such as a frame rate or frame format). The number of stream descriptors is equal to the number of streams in the multiplexed data.

[0280] When multiplexed data is recorded on a recording medium etc., it is recorded together with multiplexed data information files.

[0281] Each of the multiplexed data information files is multiplexed data management information, as shown in Fig. 51. The multiplexed data information files are in one-to-one correspondence with the multiplexed data, and each of the files contains multiplexed data information, stream attribute information, and a recording element map.

[0282] As shown in Fig. 51, the multiplexed data information includes a system rate, a playback start time, and a playback end time. The system rate indicates a maximum transmission rate at which the target system decoder described later transmits the multiplexed data to the PID filter. The ATS intervals included in the multiplexed data are set to a value not higher than the system rate. The playback start time indicates a PTS in a video frame at the beginning of the multiplexed data. An interval of one frame is added to the PTS in a video frame at the end of the multiplexed data, and the PTS is set to the playback end time.

[0283] As shown in Fig. 52, a piece of attribute information is registered in the stream attribute information for each PID of each stream included in the multiplexed data. Each piece of attribute information has different information depending on whether the corresponding stream is a video stream, an audio stream, a presentation graphics stream, or an interactive graphics stream. Each piece of video stream attribute information carries information including what type of compression codec is used to compress the video stream, and a resolution, a frame format, and a frame rate of parts of picture data that are included in the video stream. Each piece of audio stream attribute information carries information including what type of compression codec is used to compress the audio stream, how many channels are included in the audio stream, what language the audio stream supports, and how high the sampling frequency is. The video stream attribute information and the audio stream attribute information are used to initialize the decoder before the player reproduces the information.

[0284] In the present embodiment, the multiplexed data to be used is a stream type included in the PMT. Furthermore, when the multiplexed data is recorded on the recording medium, the video stream attribute information included in the multiplexed data information is used. More specifically, the moving picture coding method or the moving picture coding device described in each of the embodiments includes a step or a module for distributing unique information indicating the video data generated by the image coding method or the moving picture coding device in each of the embodiments, for the stream type included in the PMT or the video stream attribute information. With such a configuration, the video data generated by the moving picture coding method or the moving picture coding device described in each of the embodiments can be distinguished from the video data that conforms to another standard.

[0285] Furthermore, Fig. 53 illustrates the steps of the moving picture decoding method according to the present embodiment. In step exS100, the stream type included in the PMT or the video stream attribute information included in the multiplexed data information is obtained from the multiplexed data. Then, in step exS101, it is determined whether the stream type or the video stream attribute information indicates that the multiplexed data is generated by the moving picture coding method or the moving picture coding device in each of the embodiments. When it is determined that the stream type or the video stream attribute information indicates that the multiplexed data is generated by the moving picture coding method or the moving picture coding device in each of the embodiments, decoding is performed in step exS102 by the moving picture decoding method in each of the embodiments. In addition, if the stream type or attribute information of the video stream indicates conformity with conventional standards such as MPEG-2, MPEG-4 AVC, and VC-1, in step exS103, decoding is performed by the moving picture decoding method in accordance with the conventional standards.

[0286] In this way, by allocating a new unique value to the type of stream or attribute information of the video stream, it is possible to determine whether the image decoding method or the moving image decoding device described in each of the embodiments can perform decoding. Even when multiplexed data conforming to some other standard is input, an appropriate decoding method or device can be selected. In this way, it becomes possible to decode the information without any errors. Furthermore, the moving image coding method or device or the moving image decoding method or device in the present embodiment can be used in the devices and systems described above.

[0287] (Embodiment 7)

Each of the moving image encoding method, the moving image encoding device, the moving image decoding method, and the moving image decoding device in each of the embodiments is typically implemented in the form of an integrated circuit or a highly integrated circuit (LSI). As an example of the LSI, Fig. 54 illustrates a configuration of the LSI ex500, which is implemented in a single chip. The LSI ex500 includes elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509, which will be described below, and the elements are connected to each other via a bus ex510. The power supply circuit module ex505 is activated by supplying power to each of the elements when the power supply circuit module ex505 is turned on.

[0288] For example, when encoding is performed, the LSI ex500 receives an AV signal from the microphone ex117, the camera ex113 and others via the AV input/output ex509 under the control of a control unit ex501 including a CPU ex502, a memory controller ex503, a stream controller ex504 and a driving frequency control unit ex512. The received AV signal is temporarily stored in an external memory ex511 such as an SDRAM. Under the control of the control unit ex501, the stored data is segmented into data portions according to the volume and processing speed to be supplied to the signal processing unit ex507. Then, the signal processing unit ex507 encodes the audio signal and/or the video signal. Here, the encoding of the video signal is the encoding described in each of the embodiments. In addition, the signal processing unit ex507 sometimes multiplexes the encoded audio data and the encoded video data, and the stream input/output (I/O) ex506 outputs the multiplexed data to the outside. The output multiplexed data is transmitted to the base station ex107 or recorded on the recording medium ex215. If the data sets are multiplexed, these data must be temporarily stored in the buffer ex508 so that the data sets are synchronized with each other.

[0289] Although the memory ex511 is an element external to the LSI ex500, it may be included in the LSI ex500. The buffer ex508 is not limited to a single buffer, but may consist of buffers. In addition, the LSI ex500 may be implemented on a single chip or a plurality of chips.

[0290] Furthermore, although the control unit ex501 includes a CPU ex502, a memory controller ex503, a flow controller ex504, an excitation frequency control unit ex512, the configuration of the control unit ex501 is not limited thereto. For example, the signal processing unit ex507 may further include a CPU. Including another CPU in the signal processing unit ex507 can improve the processing speed. Furthermore, as another example, the CPU ex502 may serve as the signal processing unit ex507 or be a part thereof, and, for example, may include an audio signal processing unit. In such a case, the control unit ex501 includes the signal processing unit ex507, or the CPU ex502 includes a part of the signal processing unit ex507.

[0291] The term used here is LSI, but it can also be referred to as an integrated circuit (IC), a system LSI, a super LSI, or an ultra LSI depending on the degree of integration.

[0292] Furthermore, the methods for achieving integration are not limited to the LSI, and a specialized circuit or a general-purpose processor, etc. may also implement the integration. A field-programmable gate array (FPGA) that can be programmed after the LSI is manufactured, or a reconfigurable processor that allows the connection or configuration of the LSI to be changed, may be used for the same purpose.

[0293] In the future, with the advancement of semiconductor technology, a completely new technology may replace the LSI. Functional blocks may be integrated using such technology. It is also possible that the present invention will be applied to biotechnology.

[0294] (Embodiment 8)

When video data generated by the moving image encoding method or the moving image encoding device described in each of the embodiments is decoded, compared with when video data that conforms to a conventional standard such as MPEG-2, MPEG-4 AVC, and VC-1 is decoded, the amount of processing is likely to increase. Therefore, the LSI ex500 needs to be set to a driving frequency higher than that of the CPU ex502 to be used when video data conforming to a conventional standard is decoded. However, when the driving frequency is set higher, there is a problem that power consumption increases.

[0295] In order to solve this problem, a moving picture decoding device such as the television ex300 and the LSI ex500 is configured to determine which standard the video data conforms to and to switch between driving frequencies in accordance with the determined standard. Fig. 55 illustrates a configuration ex800 in the present embodiment. The driving frequency switching unit ex803 sets the driving frequency to a higher driving frequency when the video data is generated by the moving picture coding method or the moving picture coding device described in each of the embodiments. Then, the driving frequency switching unit ex803 instructs the decoding processing unit ex801, which executes the moving picture decoding method described in each of the embodiments, to decode the video data. When the video data conforms to a conventional standard, the driving frequency switching unit ex803 sets the driving frequency to a lower driving frequency than that of the video data obtained by the moving image coding method or the moving image coding device described in each of the embodiments. Then, the driving frequency switching unit ex803 instructs the decoding processing unit ex802, which conforms to the conventional standard, to decode the video data.

[0296] More specifically, the driving frequency switching unit ex803 includes a CPU ex502 and a driving frequency control unit ex512 of Fig. 54. Here, each of the decoding processing unit ex801 that executes the moving image decoding method described in each of Embodiments and the decoding processing unit ex802 that conforms to a conventional standard corresponds to the signal processing unit ex507 of Fig. 54. The CPU ex502 determines which standard the video data conforms to. Then, the driving frequency control unit ex512 determines the driving frequency based on the signal from the CPU ex502. Furthermore, the signal processing unit ex507 decodes the video data based on the signal from the CPU ex502. For example, the identification information described in Embodiment 6 can be used to identify the video data. The identification information is not limited to that described in Embodiment 6, but may be any information as long as the information indicates what standard the video data conforms to. For example, if what standard the video data conforms to can be determined based on an external signal for determining that the video data is used for a television or a disc, etc., the determination can be made based on such an external signal. In addition, the CPU ex502 selects a driving frequency based on, for example, a look-up table in which video data standards are associated with driving frequencies, as shown in Fig. 57. The driving frequency can be selected by storing the look-up table in the buffer ex508 and in the internal memory of the LSI and with reference to the look-up table by the CPU ex502.

[0297] Fig. 56 illustrates steps for executing the method in the present embodiment. First, in step exS200, the signal processing unit ex507 obtains identification information from the multiplexed data. Then, in step exS201, the CPU ex502 determines whether the video data is generated by the encoding method and the encoding device described in each of the embodiments, based on the identification information. If the video data is generated by the encoding method and the encoding device described in each of the embodiments, then, in step exS202, the CPU ex502 transmits a signal for setting the driving frequency to a higher driving frequency to the driving frequency control unit ex512. Then, the driving frequency control unit ex512 sets the driving frequency to the higher driving frequency. On the other hand, when the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG-4 AVC and VC-1, in step exS203, the CPU ex502 transmits a signal for setting the driving frequency to a lower frequency to the driving frequency control unit ex512. Then, the driving frequency control unit ex512 sets the driving frequency to a lower driving frequency than in the case where the video data is generated by the moving image coding method and the moving image coding device described in each embodiment.

[0298] In addition, in addition to switching the driving frequencies, the power saving effect can be improved by changing the voltage supplied to the LSI ex500 or the device including the LSI ex500. For example, when the driving frequency is set lower, the voltage supplied to the LSI ex500 or the device including the LSI ex500 can be set to a voltage lower than in the case where the driving frequency is set higher.

[0299] Furthermore, when the amount of processing for decoding is larger, the driving frequency can be set higher, and when the amount of processing for decoding is smaller, the driving frequency can be set lower as a method for setting the driving frequency. Thus, the setting method is not limited to the one described above. For example, when the amount of processing for decoding video data according to MPEG-4 AVC is larger than the amount of processing for decoding video data generated by the moving picture coding method and the moving picture coding device described in each of the embodiments, the driving frequency can be set in the reverse order of the setting described above.

[0300] Furthermore, the method for setting the driving frequency is not limited to the method for setting the driving frequency lower. For example, when the identification information indicates that the video data is generated by the moving image coding method and the moving image coding device described in each of the embodiments, the voltage supplied to the LSI ex500 or the device including the LSI ex500 may be set higher. When the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG-4 AVC and VC-1, the voltage supplied to the LSI ex500 or the device including the LSI ex500 may be set lower. As another example, when the identification information indicates that the video data is generated by the moving image coding method and the moving image coding device described in each of the embodiments, the operation of the CPU ex502 may not need to be suspended. When the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG-4 AVC and VC-1, the operation of the CPU ex502 is likely to be suspended at this point in time because the CPU ex502 has additional processing capacity. Even when the identification information indicates that the video data is generated by the moving image coding method and the moving image coding device described in each of the embodiments, in the case where the CPU ex502 has additional processing capacity, the operation of the CPU ex502 may be suspended at this point in time. In such a case, the pause time is likely to be set to a smaller value than in the case where the significant_flag identification information indicates that the video data complies with a common standard, such as MPEG-2, MPEG-4 AVC, and VC-1.

[0301] Accordingly, the power saving effect can be improved by switching between driving frequencies in accordance with the standard to which the video data conforms. In addition, when the LSI ex500 or a device including the LSI ex500 is driven using a battery, the life of the battery can be extended due to the power saving effect.

[0302] (Embodiment 9)

There are cases where a plurality of video data that conforms to different standards are supplied to devices and systems such as a television and a cellular phone. In order to decode a plurality of video data that conforms to different standards, the signal processing module ex507 of the LSI ex500 must conform to different standards. However, there are problems of increasing the integration degree of the LSI ex500 and increasing the cost when using the signal processing modules ex507 that conform to the corresponding standards individually.

[0303] In order to solve this problem, a configuration is proposed in which a decoding processing unit for implementing the moving picture decoding method described in each of the embodiments and a decoding processing unit that conforms to a conventional standard such as MPEG-2, MPEG-4 AVC and VC-1 are partially shared. Ex900 in Fig. 58A shows an example of such a configuration. For example, the moving picture decoding method described in each of the embodiments and the moving picture decoding method that conforms to MPEG-4 AVC have partially common processing details such as entropy coding, inverse quantization, deblocking filtering and motion compensated prediction. The processing details for shared use may include using the decoding processing unit ex902 that conforms to MPEG-4 AVC. In contrast, the dedicated decoding processing unit ex901 may be used for other processing that is unique to the aspect of the present invention and does not correspond to MPEG-4 AVC. Since the aspect of the present invention is characterized by entropy decoding, in particular, for example, the dedicated decoding processing unit ex901 is used for entropy decoding. Otherwise, the decoding processing unit may be shared for one of inverse quantization, deblocking filtering and motion compensation or all processing. The decoding processing unit for implementing the moving picture decoding method described in each of the embodiments may be shared for shared processing, and the dedicated decoding processing unit may be used for processing unique to MPEG-4 AVC.

[0304] Furthermore, ex1000 in Fig. 58B shows another example that the processing is partially divided. This example uses a configuration including a dedicated decoding processing unit ex1001 that supports the processing unique to an aspect of the present invention, a dedicated decoding processing unit ex1002 that supports the processing unique to another conventional standard, and a decoding processing unit ex1003 that supports the processing shared between the moving picture decoding method according to an aspect of the present invention and the conventional moving picture decoding method. Here, the dedicated decoding processing units ex1001 and ex1002 are not necessarily specialized for the processing according to the aspect of the present invention and the processing of the conventional standard, respectively, and may be any capable of implementing the common processing. Furthermore, the configuration of the present embodiment can be implemented by the LSI ex500.

[0306] Thus, a reduction in the degree of integration of the LSI and a reduction in cost are possible by sharing the decoding processing module for the processing shared between the moving image decoding method according to an aspect of the present invention and the moving image decoding method according to a conventional standard.

[0306] Industrial applicability

The present invention can be applied to image encoding methods, image decoding methods, image encoding devices and image decoding devices. The present invention is also applicable to high-resolution information display devices or image forming devices, such as televisions, digital video recorders, car navigation systems, portable telephones, digital cameras, each of which includes an image encoding device.

List of reference positions

[0307]

100 image encoding device

101 Coding Block Division Module

102 subtraction module

103 Conversion Module

104 Variable Length Coding Module

105, 202 inverse transform module

106, 203 summation module

107, 205 frame memory

108 prediction module

121 input images

122 coding block

123, 126, 223 differential block

124, 222 frequency coefficient (coefficient)

125, 221 coded stream

127, 224 decoded block

128, 225, 226 decoded image

129 prediction block

131 module of division into blocks of coefficients

132 module encoding significant_flag

133, 133A level coding module

134 coding module sign_flag

141 module install greater1_flag

142 greater2_flag installation module

143, 143A residual installation module

144, 248 binarization parameter update module

145, 145A, 241, 241A context set selection module

146, 243 memory greater1_flag

147, 242 context selection module greater1_flag

148, 244 greater1_flag context memory

149, 245 context selection module greater2_flag

150, 246 greater2_flag context memory

151 arithmetic coding module

152 Residue Binarization Module

153 Prefix-Suffix Definition Module

154 suffix bin output module

155 Bin Prefix Output Module

160, 260 greater_flag switching module

200 image decoding device

201 variable length decoding module

204 decoded block merging module

231 decoding module significant_flag

232, 232A level decoding module

233 sign_flag decoding module

234 Coefficient Decoding Module

247, 247A arithmetic decoding module

249 module of multi-valued remainder

250 Prefix Decoding Module

251 Suffix Decoding Module

252 Prefix-Suffix Merge Module

Claims (13)

1. A method for transmitting a bit stream over a network, the method comprising: transmission of a bit stream over a network, where the bit stream is intended to encode the current block and is generated by: determining whether an immediately preceding block of coefficients that was encoded immediately before the target block of coefficients includes a coefficient that has an absolute value greater than a threshold value; selecting a first context to be used in arithmetic encoding of a first flag of a target coefficient in a target block of coefficients, wherein the selection of the context to be used in arithmetic encoding of the first flag is based at least in part on a result of determining whether an immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value; arithmetic encoding of the first flag using the first context; selecting a second context to be used in arithmetic encoding of a second flag of a target coefficient in a target block of coefficients, wherein the selection of the context to be used in arithmetic encoding of the second flag is based, at least in part, on the result of determining whether an immediately preceding block of coefficients includes a coefficient having an absolute value greater than said threshold value; and arithmetic encoding of the second flag using the second context; in this case the threshold value is equal to 1. 2. The method according to claim 1, wherein the first flag indicates whether the absolute value of the target coefficient in the target block of coefficients is greater than 1. 3. The method according to claim 1, wherein the second flag indicates whether the absolute value of the target coefficient in the target block of coefficients is greater than 2. 4. The method of claim 1, wherein determining whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value is based at least in part on a variable generated after arithmetic coding over the immediately preceding block of coefficients. 5. The method according to claim 1, further comprising binarizing the residual according to a lookup table designated by the binarization parameter from among the plurality of lookup tables when the absolute value of the target coefficient is greater than 2, wherein the residual is a value obtained by subtracting 3 from the absolute value. 6. The method according to claim 5, wherein the binarization parameter is determined according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value that is greater than said threshold value, and wherein said variable is a binarization parameter.

Images