RU2824877C1 - Method of decoding images - Google Patents

Abstract

FIELD: means for decoding images.

SUBSTANCE: quantized block parameter is decoded. Inverse quantisation and inverse transformation of the quantised parameter are performed to form the reconstructed residual block. SAO type parameter is decoded from the bitstream using constant-probability bypass arithmetic decoding. SAO type parameter indicates whether the SAO value is a boundary shift value or a range shift value, and the SAO value is to be added to the pixel value of the reconstructed image. Plurality of bins is decoded from the bitstream using arithmetic bypass decoding with a constant probability, wherein the plurality of bins is an integer indicating the value of the SAO value. An integer indicating the value of the SAO value is determined based on a plurality of bins decoded using arithmetic bypass coding with a constant probability. Value of SAO is determined using the decoded parameter of the SAO type and said determined integer. Decoded block is summed up with said determined SAO value.

EFFECT: high efficiency of decoding images.

1 cl, 38 dwg

Description

[Field of technology]

[0001] The present invention relates to an image encoding method, an image encoding device, an image decoding method, an image decoding device, and an image encoding and decoding device, and in particular, to an image encoding method, an image encoding device, an image decoding method, an image decoding device, and an image encoding and decoding device for performing arithmetic encoding or arithmetic decoding on a sample adaptive offset (SAO) parameter.

[Prior Art]

[0002] Recently, there has been significant technical development in digital video devices, and increasing opportunities for encoding a video signal (multiple images organized into a time series) with compression and recording the video signal on recording media such as DVDs and hard disks, or distributing the video signal on the Internet. For example, H.264/AVC (MPEG-4 AVC) is one of the standards for encoding a video signal with compression. In addition, the High Efficiency Video Coding (HEVC) standard is currently being considered as the next generation standard.

[0003] The current HEVC standard (NPL 1) contains an offset process called SAO. The SAO process is a process for adding an offset value to each pixel of an image decoded from a bitstream (coded bitstream). Performing the SAO process makes it possible to reduce the degradation of the image as a result of encoding and make the decoded image more similar to the original image. According to the current HEVC standard, the SAO process performs arithmetic coding on the offset value and adds the offset value to the bitstream.

[List of bibliographic references]

[Non-patent literature]

[0004] [NPL 1] Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 9th Meeting: Geneva, CH, 27 April-7 May 2012, JCTVC- I0602_CDTexts_r2.doc, BoG report on integrated text of SAO adoptions on top of JCTVC-I0030, http://phenix.it-sudparis.eu/jct/doc_end_user/documents/9_Geneva/wg11/JCTVC-I0602-v3.zip

[Essence of the invention]

[Technical task]

[0005] In the traditional method, accelerated processing is desired when encoding and decoding an image.

[0006] Here, the present invention provides an image encoding method and an image decoding method in which processing can be accelerated.

[Problem solution]

[0007] An image encoding method according to an aspect of the present invention is an image encoding method for encoding an input image and includes: converting into a bin string an offset value used in a offset process that is to be applied to a pixel value of a reconstructed image corresponding to the input image; and performing bypass arithmetic encoding on the bin string using a constant probability.

[0008] Furthermore, an image decoding method according to another aspect of the present invention is an image decoding method for decoding an encoded image and includes: performing bypass arithmetic decoding on a bin row using a constant probability, wherein the bin row is encoded and indicates an offset value used in the offset process, which should be applied to a pixel value of a reconstructed image corresponding to the encoded image; and converting into an offset value the bin row on which the bypass arithmetic decoding is performed.

[0009] These general or specific aspects of the present invention may be implemented using a system, device, integrated circuit, computer program, computer-readable recording medium such as a CD-ROM, or any combination of a system, device, integrated circuit, computer program, and recording medium.

[Useful results of the invention]

[0010] By using the image encoding method, the image encoding device, the image decoding method, the image decoding device and the image encoding and decoding device according to the present invention, it is possible to accelerate processing.

[Brief description of the drawings]

[0011] Fig. 1 is a block diagram illustrating an exemplary configuration of an image encoding device according to Embodiment 1.

Fig. 2 is a block diagram illustrating an exemplary configuration of a SAO parameter encoding unit.

Fig. 3 is a block diagram illustrating an exemplary configuration of the sao_offset encoding node.

Fig. 4 is a flow chart showing exemplary processes in the image encoding device according to Embodiment 1.

Fig. 5 is a flow chart showing exemplary processes in the SAO parameter encoding unit according to Embodiment 1.

Fig. 6A is a flow chart showing exemplary processes in the sao_offset encoding node according to Embodiment 1.

Fig. 6B is a flow chart showing exemplary processes in the sao_offset encoding node according to Variation 2 of Embodiment 1.

Fig. 7 is a table showing the approximate correspondence between non-binary signals and bin rows.

Fig. 8 is a table indicating the correspondence between the bins included in binIdx and the arithmetic coding to be applied.

Fig. 9 is a table showing the result of an experiment on the coding efficiency according to Embodiment 1 and Variation 1.

Fig. 10 is a block diagram illustrating an exemplary configuration of an image decoding device according to Embodiment 2.

Fig. 11 is a block diagram illustrating an exemplary configuration of a SAO parameter decoding node.

Fig. 12 is a block diagram illustrating an exemplary configuration of the sao_offset decoding node.

Fig. 13 is a flow chart showing exemplary processes in the image decoding device according to Embodiment 2.

Fig. 14 is a flow chart showing exemplary processes in the SAO parameter decoding unit according to Embodiment 2.

Fig. 15A is a flow chart showing exemplary processes in the sao_offset decoding node according to Embodiment 2.

Fig. 15B is a flow chart showing exemplary processes in the sao_offset decoding node according to Variation 2 of Embodiment 2.

Fig. 16 illustrates a general configuration of a content provision system for implementing content distribution services.

Fig. 17 illustrates the general configuration of the digital broadcasting system.

Fig. 18 illustrates a block diagram of an example configuration of a television.

Fig. 19 is a block diagram illustrating an example configuration of an information reproducing/recording unit that reads and writes information from or to a recording medium that is an optical disc.

Fig. 20 illustrates an example of a configuration of a recording medium which is an optical disc.

Fig. 21A illustrates an example of a cellular telephone.

Fig. 21B illustrates a block diagram of an example configuration of a cellular telephone.

Fig. 22 illustrates the structure of multiplexed data.

Fig. 23 schematically illustrates how each stream is multiplexed into multiplexed data.

Fig. 24 illustrates in more detail how the video stream is stored in the PES packet stream.

Fig. 25 illustrates the structure of TS packets and source packets in multiplexed data.

Fig. 26 illustrates the PMT data structure.

Fig. 27 illustrates the internal structure of the multiplexed data information.

Fig. 28 illustrates the internal structure of the flow attribute information.

Fig. 29 illustrates steps for identifying video data.

Fig. 30 illustrates a block diagram illustrating an example of a configuration of an integrated circuit for implementing a movie encoding method and a movie decoding method according to each of the Embodiments.

Fig. 31 illustrates a configuration for switching between excitation frequencies.

Fig. 32 illustrates the steps for identifying video data and switching between excitation frequencies.

Fig. 33 illustrates an example of a look-up table in which video data standards are associated with excitation frequencies.

Fig. 34A illustrates an example of a configuration for sharing a module in a signal processing node.

Fig. 34B illustrates an example of a configuration for sharing a module in a signal processing node.

[Description of embodiments]

[0012] (Information on which the present invention is based)

The inventors have found that the image coding method and others described in the "Prior Art" have the following problems.

[0013] As described above, according to the current HEVC standard, in the SAO process, which is an offset process, arithmetic coding is performed on an offset value, and the offset value is added to a binary signal stream. More specifically, in the arithmetic coding of the HEVC standard, a signal to be encoded (for example, an SAO offset value) is converted from a non-binary signal into a bin string, and arithmetic coding is performed on the bin string.

[0014] Arithmetic coding has two types: (1) contextual arithmetic coding, in which arithmetic coding is performed using an adaptive probability of occurrence of a symbol; and (2) bypass arithmetic coding, in which arithmetic coding is performed by setting the probability of occurrence of a symbol to, for example, 50%.

[0015] (1) In context arithmetic coding, a context is selected for each signal to be coded. The probability of occurrence of a symbol is determined in accordance with that context. In addition, in context arithmetic coding, the selected context is loaded, and arithmetic coding is performed on the signal to be coded based on the probability of occurrence of a symbol corresponding to the loaded context. In addition, in context arithmetic coding, a context update process for updating the probability of occurrence of a symbol (for example, the probabilities of occurrence of 1) of the context is performed in accordance with the value of the signal to be coded.

[0016] (2) In contrast, in bypass arithmetic coding, arithmetic coding is performed on the signal to be coded by setting the probability of occurrence of a symbol to, for example, 50% without using any context. Thus, bypass arithmetic coding does not require loading and updating context.

[0017] Conventionally, an image encoding device performs context arithmetic coding on an SAO offset value. The reason is that the probability of occurrence of a symbol of a bin string indicating an SAO offset value is likely to have some deviation according to the HEVC standard (NPL 1). When the probability of occurrence of a symbol has a deviation, the decrease in coding efficiency can be stopped by using context arithmetic coding.

[0018] However, context arithmetic coding requires processing such as loading and updating a context. In addition, when a context identical to the context selected in context arithmetic coding over a predetermined SAO offset value is selected in context arithmetic coding over another SAO offset value, context arithmetic coding over the next SAO offset value cannot be started until the context for the SAO offset value over which arithmetic coding is performed first is updated. Thus, problems with sequential processes and a decrease in throughput occur.

[0019] In order to solve such problems, an image encoding method according to an aspect of the present invention is an image encoding method for encoding an input image and includes: converting into a bin string an offset value used in a offset process that is to be applied to a pixel value of a reconstructed image corresponding to the input image; and performing bypass arithmetic encoding on the bin string using a constant probability.

[0020] Since in the image encoding method, arithmetic encoding is performed on the SAO offset value using bypass arithmetic encoding, the processing can be accelerated.

[0021] The inventors have found that the probability of occurrence of symbols in the bin strings indicating the SAO offset value has small deviations, which will be described later. Therefore, the encoding efficiency is unlikely to decrease even when bypass arithmetic encoding is performed. Therefore, the inventors have confirmed through an experiment that bypass arithmetic encoding as arithmetic encoding over the SAO offset value does not decrease the encoding efficiency.

[0022] Thus, the method of encoding images with this configuration makes it possible to speed up processing without reducing the encoding efficiency.

[0023] Furthermore, an image decoding method according to another aspect of the present invention is an image decoding method for decoding an encoded image and includes: performing bypass arithmetic decoding on a bin row using a constant probability, wherein the bin row is encoded and indicates an offset value used in the offset process, which should be applied to a pixel value of a reconstructed image corresponding to the encoded image; and converting into an offset value the bin row on which the bypass arithmetic decoding is performed.

[0024] Furthermore, an image encoding device according to another aspect of the present invention is an image encoding device that encodes an input image and includes a control circuit and a memory device electrically connected to the control circuit, wherein the control circuit executes: converting into a bin string an offset value used in a offset process that is to be applied to a pixel value of a reconstructed image corresponding to the input image; and performing bypass arithmetic encoding on the bin string using a constant probability.

[0025] Furthermore, an image decoding device according to another aspect of the present invention is an image decoding device that decodes an encoded image and includes a control circuit and a memory device electrically connected to the control circuit, wherein the control circuit executes: performing bypass arithmetic decoding on a bin row using a constant probability, wherein the bin row is encoded and indicates an offset value used in the offset process, which should be applied to a pixel value of a reconstructed image corresponding to the encoded image; and converting into an offset value the bin row on which the bypass arithmetic decoding is performed.

[0026] Furthermore, an image encoding and decoding device according to another aspect of the present invention includes an image encoding device and an image decoding device.

[0027] These general or specific aspects of the present invention may be implemented using a system, method, integrated circuit, computer program, computer-readable recording medium such as a CD-ROM, or any combination of a system, method, integrated circuit, computer program, and recording medium.

[0028] Embodiments according to the present invention will be described with reference to the drawings.

[0029] The embodiments described below indicate specific or general examples of the present invention. The values, shapes, materials, constituent elements, positions and connections of the constituent elements, steps and orders of steps indicated in the Embodiments are examples and do not limit the present invention. Constituent elements in the Embodiments that are not described in the independent claims that describe the most general idea of the present invention are described as arbitrary constituent elements.

[0030] (Implementation option 1)

An image encoding device and an image encoding method according to Embodiment 1 will be described with reference to Figs. 1-9.

[0031] <1-1. Device Configuration>

An image encoding device according to Embodiment 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram illustrating the configuration of an image encoding device 100 according to the HEVC standard according to Embodiment 1.

[0032] The image encoding device 100 is a device that encodes an image to be input (input image) block by block to form a binary signal stream (coded binary signal stream).

[0033] As illustrated in Fig. 1, the image encoding device 100 includes a block division unit 101, a subtraction unit 102, a transformation unit 103, a variable length encoding unit (entropy encoding unit) 104, an inverse transformation unit 105, an addition unit 106, an SAO processing unit 107, an SAO parameter encoding unit 110, a frame memory 108, and a prediction unit 109.

[0034] The block division unit 101 divides the input image into blocks that are to be predicted and include pixels. The blocks that are to be predicted are blocks of variable size. For example, the minimum size of the blocks is 4×4 pixels, and the maximum size of the blocks is 32×32 pixels.

[0035] The subtraction unit 102 subtracts the prediction block predicted by the prediction unit 109, which will be described later, from each of the blocks to be predicted to calculate a difference block. The transformation unit 103 transforms the difference block from the image domain to the frequency domain to output frequency coefficients, and quantizes these frequency coefficients. The quantized frequency coefficients will be referred to as quantized coefficients hereinafter.

[0036] The variable length encoding unit 104 performs entropy encoding on the quantized coefficients output from the transform unit 103 and various information necessary for decoding (partition information for blocks to be predicted, prediction type, motion vector, prediction mode (intra prediction mode), quantized parameter, etc.) to generate a binary signal stream.

[0037] The inverse transform unit 105 performs inverse quantization to transform the quantized coefficients output from the transform unit 103 into frequency coefficients and performs inverse transform to transform the frequency coefficients from the frequency domain to the image domain to output a reconstructed difference block. The addition unit 106 adds the prediction block predicted by the prediction unit 109, which will be described later, with the reconstructed difference block output by the inverse transform unit 105 to form a reconstructed block.

[0038] The SAO processing unit 107 performs an SAO process (offset process) on the reconstructed block to add to the pixel value of each pixel included in the reconstructed block an SAO offset value corresponding to the category (class) to which the pixel belongs, which will be described later. Since the process performed by the conversion unit 103 is an irreversible process, an error occurs between each of the blocks to be predicted in the input image and the reconstructed block. The SAO offset value is a value for correcting the error and is set for each category.

[0039] The SAO parameter encoding unit 110 performs variable length encoding on various parameters to be used in the SAO process, and adds the parameters to the bit stream output from the variable length encoding unit 104. Embodiment 1 specifies sao_offset indicating an SAO offset value and sao_type_idx indicating a category of the SAO offset value as SAO parameters.

[0040] Fig. 2 is a block diagram illustrating an exemplary configuration of an SAO parameter encoding unit 110. As illustrated in Fig. 2, the SAO parameter encoding unit 110 includes an sao_type_idx encoding unit 111 and a sao_offset encoding unit 112.

[0041] The sao_type_idx encoding unit 111 performs arithmetic encoding on the sao_type_idx indicating the category of the sao_offset. Here, the sao_offset is classified into two categories: a boundary offset value determined according to the state of the surrounding pixels; and a range offset value determined according to the region to which the pixel value of the pixel belongs.

[0042] The sao_offset encoding unit 112 performs arithmetic encoding on the sao_offset. Fig. 3 is a block diagram illustrating an exemplary configuration of the sao_offset encoding unit 112.

[0043] As illustrated in Fig. 3, the sao_offset encoding unit 112 includes a sao_offset to binary conversion unit 113, an arithmetic encoding switching unit 114, a context 0 arithmetic encoding unit 115, a context 1 arithmetic encoding unit 116, and a bypass arithmetic encoding unit 117.

[0044] The sao_offset to binary form conversion unit 113 converts the sao_offset, which is a non-binary signal, into a binary form (converts the sao_offset into a signal representing 0 and 1). The arithmetic coding switching unit 114 switches the arithmetic coding method to be applied to the sao_offset. The context 0 arithmetic coding unit 115 performs arithmetic coding of context 0 using the first context (context 0). The context 1 arithmetic coding unit 116 performs arithmetic coding of context 1 using the second context (context 1). The bypass arithmetic coding unit 117 performs bypass arithmetic coding using a constant probability.

[0045] The frame memory 108 includes some storage device such as random access memory (RAM) and read-only memory (ROM).

[0046] The prediction unit 109 generates a prediction block by means of inter prediction or intra prediction. In the inter prediction, the prediction unit 109 outputs a motion vector using a block to be predicted, and generates a prediction block using the output motion vector and a reconstructed block obtained in the SAO process and stored in the frame memory 108. In the intra prediction, the prediction unit 109 generates a prediction block using a reconstructed block obtained in the SAO process and stored in the frame memory 108, according to a prediction mode.

[0047] <1-2 General Operations>

Next, operations of the image encoding device will be described with reference to Fig. 4. Fig. 4 is a flow chart indicating a procedure in the image encoding device 100.

[0048] In the image encoding device 100, the block division unit 101 first divides the input image into blocks to be predicted and sequentially outputs these blocks to the subtraction unit 102 and the prediction unit 109 (step S101). The blocks to be predicted have a variable size according to Embodiment 1. The input image is divided using image features. The minimum size of the blocks to be predicted is 4 horizontal pixels × 4 vertical pixels according to Embodiment 1. In addition, the maximum size of the blocks to be predicted is 32 horizontal pixels × 32 vertical pixels according to Embodiment 1.

[0049] The prediction unit 109 in the image encoding device 100 generates a prediction block using the blocks to be predicted and the reconstructed block stored in the frame memory 108 (step S102).

[0050] The subtraction unit 102 in the image encoding device 100 subtracts a prediction block from each of the blocks to be predicted to form a difference block (step S103).

[0051] The conversion unit 103 in the image encoding device 100 converts the difference block into frequency coefficients and converts the frequency coefficients into quantized coefficients (step S104).

[0052] The inverse transform unit 105 in the image encoding device 100 inversely quantizes the quantized coefficients to output frequency coefficients, and transforms the frequency coefficients from the frequency domain to the image domain to generate a reconstructed difference block (step S105).

[0053] The addition unit 106 in the image encoding device 100 adds the prediction block with the reconstructed difference block to form a reconstructed block (step S106).

[0054] The SAO processing section 107 in the image encoding device 100 performs the SAO process (step S107). The SAO processing section 107 first determines the SAO parameters to be used for the SAO process. In Embodiment 1, the SAO parameters are sao_offset indicating an SAO offset value and sao_type_idx indicating a category of the SAO offset value. The SAO processing section 107 adds the SAO offset value to a pixel value of each pixel included in the reconstructed block, and stores the reconstructed block to which the SAO offset values are added in the frame memory 108. In the SAO process, each of the pixels included in the reconstructed block is classified into categories, and an SAO offset value is output for each of those categories. There are numerous methods for classifying pixels. Thus, the sao_type_idx parameter indicating the category of the SAO offset value is used.

[0055] The SAO parameter encoding unit 110 in the image encoding device 100 performs variable length encoding on the SAO parameter and adds the parameter to the binary signal stream output from the variable length encoding unit 104 (step S108). Details of the SAO parameter encoding unit 110 will be described later.

[0056] The variable length encoding unit 104 in the image encoding device 100 performs variable length encoding on the quantized coefficients output from the transform unit 103 to output a binary signal stream (step S109).

[0057] The image encoding device 100 repeatedly performs Steps S102 to S109 until encoding of all blocks of the input image to be predicted is completed (Step S110).

[0058] <1-2-1. SAO Parameter Coding Node Operations>

The operations of the SAO parameter encoding unit 110 according to Embodiment 1 will be described with reference to Fig. 5.

[0059] Fig. 5 is a flow chart indicating exemplary processes of the SAO parameter encoding unit 110, and describes in detail Step S108 of Fig. 4.

[0060] The sao_type_idx encoding unit 111 in the SAO parameter encoding unit 110 encodes the sao_type_idx (step S111).

[0061] The sao_offset encoding unit 112 in the SAO parameter encoding unit 110 encodes the sao_offset (step S112).

[0062] <1-2-2. sao_offset encoding node operations>

The operations of the sao_offset encoding unit 112 according to Embodiment 1 will be described with reference to Fig. 6A. Fig. 6A is a flow chart indicating exemplary processes in the sao_offset encoding unit 112.

[0063] The sao_offset to binary conversion unit 113 in the sao_offset encoding unit 112 converts the sao_offset from a non-binary signal into a bin string (step S121).

[0064] Embodiment 1 describes a case where sao_offset takes values from 0 to 7. Fig. 7 is a table indicating the correspondence between non-binary signals and bin rows. As can be seen from Fig. 7, the number of consecutive "1"s from the beginning of each bin row is equal to the value indicated by the non-binary signal. In addition, "0" is added to the end when the non-binary signal is between 0 and 6. When the non-binary signal is 7, that is, the maximum number, no "0" is added to the end. Here, binIdx is an index of a bin (one digit) included in the bin row. The first value of binIdx is 0, and subsequent values are increased by 1. The bin row is output to the arithmetic coding switching unit 114 sequentially from the bin whose binIdx is 0.

[0065] The arithmetic coding switching unit 114 in the sao_offset coding unit 112 determines the value of binIdx (steps S122, S124 and S126).

[0066] When the value is 0 (Yes in step S122), the arithmetic encoding switching unit 114 outputs the bin to the context 0 arithmetic encoding unit 115. The context 0 arithmetic encoding unit 115 performs context arithmetic encoding using context 0 (step S123).

[0067] When the value is 1 (No in step S122 and Yes in step S124), the arithmetic encoding switching unit 114 outputs the bin to the context 1 arithmetic encoding unit 116. The context 1 arithmetic encoding unit 116 performs context arithmetic encoding using context 1 (step S125).

[0068] When the value is neither 0 nor 1 (NO in step S122 and NO in step S124), the arithmetic coding switching unit 114 outputs the bin to the bypass arithmetic coding unit 117. The bypass arithmetic coding unit 117 performs bypass arithmetic coding (step S126).

[0069] Since the bypass arithmetic encoding unit 117 does not use any context, the sao_offset encoding unit 112 generally uses two types of contexts according to Embodiment 1.

[0070] The sao_offset encoding unit 112 repeats the processes from Step 122 to Step 127 until the processing of all bins included in the bin string is completed (step S127).

[0071] <1-3. Advantages, varieties, etc.>

The image encoding device 100 according to Embodiment 1 can accelerate the processing by performing bypass arithmetic encoding on the bins in the second half of sao_offset. More specifically, according to Embodiment 1, not context arithmetic encoding but bypass arithmetic encoding is performed on the bins whose binIdx is 2 or higher. As described above, the bypass arithmetic encoding does not require loading and updating any context. In this way, arithmetic encoding on the SAO offset value of the block to be predicted can be started without waiting for the completion of arithmetic encoding on another SAO offset value, and the processing can be accelerated.

[0072] In addition, the probabilities of occurrence of symbols (probabilities of occurrence of 1) of binIdx 1 through 6 according to the HEVC standard (NPTL 1) that is currently being studied are almost the same, but not equal to 50%, and have a deviation. In other words, when there is binIdx after binIdx1 (when sao_offset is 1 or higher), the cases are often divided into the case where binIdx1 is 0 without any binIdx2 and after (sao_offset is 1), and the case where all binIdx 1 through 6 are 1 (sao_offset is 7).

[0073] However, the inventors found that as a result of the experiment, for encoding binIdx 1 to 6 with a constant probability of character occurrence of 50%, the encoding efficiency is almost not deteriorated. In particular, the experiment showed that sao_offset often indicates an intermediate value (from 2 to 6), and the probabilities of character occurrence of binIdx 1 to 6 are close to 50%. Therefore, the processing for suppressing the decrease in encoding efficiency can be accelerated by using not context arithmetic coding but bypass arithmetic coding on binIdx after binIdx2.

[0074] <1-3-1. Varieties>

Although bypass arithmetic coding is performed on the bins after binIdx2 according to Embodiment 1, the processing is not limited to this.

[0075] Fig. 8 is a table indicating the correspondence between bins included in binIdx and the arithmetic coding to be applied. In Fig. 8, "Embodiment 1" indicates the correspondence between the bins described in Embodiment 1 and the arithmetic coding to be applied. In addition, "Context 0" indicates context arithmetic coding using context 0. "Context 1" indicates context arithmetic coding using context 1. "Bypass" indicates bypass arithmetic coding.

[0076] "Variation 1" in Fig. 8 indicates Variation 1. According to Variation 1, context arithmetic coding is performed on the bin in binIdx0, and bypass arithmetic coding is performed on the bins after binIdx1. In addition, according to Variation 1, the sao_offset coding unit 112 in Fig. 2 should include only the sao_offset binarization unit 113, the arithmetic coding switch unit 114, the context 0 arithmetic coding unit 115, and the bypass arithmetic coding unit 117. In other words, the sao_offset coding unit 112 should not include the context 1 arithmetic coding unit 116.

[0077] "Variation 2" in Fig. 8 indicates Variation 2. In Variation 2, bypass arithmetic coding is performed on each of the bins in binIdx, as shown in Fig. 8.

[0078] Fig. 6B is a flow chart indicating exemplary processes (image encoding method) in the sao_offset encoding unit 112 according to Varieties 2. In Fig. 6B, the sao_offset is converted into a binary form (step S121 for converting the offset value into a bin string), and bypass arithmetic encoding is performed on all bins included in the sao_offset (steps S126 and S127 for performing the bypass arithmetic encoding). Since the bypass arithmetic encoding is performed on all bins, the processing can be the fastest in Varieties 2.

[0079] In addition, according to Variation 2, the sao_offset encoding unit 112 in Fig. 2 should include only the sao_offset binarization unit 113 and the bypass arithmetic encoding unit 117. In other words, the sao_offset encoding unit 112 should not include the context 0 arithmetic encoding unit 115 and the context 1 arithmetic encoding unit 116.

[0080] Although not illustrated, it is acceptable to perform context arithmetic coding on binIdx 0 through 2 and bypass arithmetic coding on binIdx after binIdx3.

[0081] <1-3-2. Coding Efficiency Experiment Result>

Fig. 9 is a table showing the result of an experiment on the coding efficiency according to Embodiment 1 and Variation 1.

[0082] Fig. 9 shows the result of a comparison between the coding efficiency obtained by performing arithmetic coding on the SAO offset value according to the current HEVC standard (NPL 1) and the coding efficiency according to Embodiment 1 and Varieties 1. The experimental conditions adhere to the general experimental conditions in the structure of the HEVC standard.

[0083] The higher the value, the lower the coding efficiency. A negative value indicates an increase in coding efficiency.

[0084] As shown in "Embodiment 1" of Fig. 9, the values indicating the comparison result fluctuate between -0.1% and 0.1%. This indicates that the encoding efficiency for encoding binIdx2 - binIdx6 of the SAO offset value is almost the same between the context arithmetic coding and the bypass arithmetic coding.

[0085] Although the coding efficiency of Variation 1 is lower than that of Embodiment 1, as can be seen from “Change 1” in Fig. 9, the results fluctuate between 0 and 0.1%, and the coding efficiency is almost the same.

[0086] Although the coding efficiency of Variation 2 is also lower than that of Embodiment 1, as can be seen from “Change 2” in Fig. 9, the results fluctuate between 0 and 0.1%, and the coding efficiency is almost the same.

[0087] As a result of the experiment on coding efficiency, the use of bypass arithmetic coding when performing arithmetic coding on the SAO offset value makes it possible to speed up arithmetic coding while making the coding efficiency less likely to decrease.

[0088] (Implementation option 2)

An image decoding device and an image decoding method according to Embodiment 2 will be described with reference to Figs. 10-15A.

[0089] <2-1. Device Configuration>

An image decoding device according to Embodiment 2 will be described with reference to Fig. 10. Fig. 10 is a block diagram illustrating the configuration of an image decoding device 200 according to the HEVC standard according to Embodiment 2.

[0090] The image decoding device 200 is a device that decodes an input binary signal stream (coded binary signal stream) to generate a decoded image (“Image” in Fig. 10).

[0091] As illustrated in Fig. 10, the image decoding device 200 includes a variable length decoding unit 201, an inverse transform unit 202, an SAO parameter decoding unit 210, an SAO processing unit 204, a decoded block combining unit 205, a frame memory 206, and a prediction unit 207.

[0092] The variable length decoding unit 201 performs entropy decoding (variable length decoding) on the binary signal stream input to the image decoding device 200 to obtain quantized coefficients and various information necessary for decoding (partition information for blocks to be predicted, prediction type, motion vector, prediction mode (intra prediction mode), quantized parameter, etc.).

[0093] The inverse transform unit 202 performs inverse quantization to transform the quantized coefficients output from the variable length decoding unit 201 into frequency coefficients and performs inverse transform to transform the frequency coefficients from the frequency domain to the image domain to output a difference block.

[0094] The addition unit 203 adds the prediction block output from the prediction unit 207 with the difference block output from the inverse transform unit 202 to form a decoded block.

[0095] The SAO parameter decoding unit 210 decodes various parameters to be used in the SAO process from a binary signal stream. Embodiment 2 specifies sao_offset indicating an SAO offset value and sao_type_idx indicating a category of an SAO offset value as SAO parameters.

[0096] Fig. 11 is a block diagram illustrating an exemplary configuration of an SAO parameter decoding unit 210. As illustrated in Fig. 11, the SAO parameter decoding unit 210 includes a sao_type_idx decoding unit 211 that performs arithmetic decoding on sao_type_idx, and a sao_offset decoding unit 212 that performs arithmetic decoding on sao_offset.

[0097] Fig. 12 is a block diagram illustrating an exemplary configuration of the sao_offset decoding unit 212. As illustrated in Fig. 12, the sao_offset decoding unit 212 includes an arithmetic decoding switching unit 213, a context 0 arithmetic decoding unit 214, a context 1 arithmetic decoding unit 215, a bypass arithmetic decoding unit 216, and an inverse sao_offset conversion unit 217 from binary form.

[0098] The arithmetic decoding switching unit 213 switches the method of arithmetic decoding to be applied to the binary signal stream. The context 0 arithmetic decoding unit 214 performs arithmetic decoding of context 0 using the first context (context 0). The context 1 arithmetic decoding unit 215 performs arithmetic decoding of context 1 using the second context (context 1). The bypass arithmetic decoding unit 216 performs bypass arithmetic decoding using a constant probability. The inverse sao_offset conversion node 217 from binary form inversely converts from binary form the string of sao_offset bins output from each of the context 0 arithmetic decoding node 214, the context 1 arithmetic decoding node 215, and the bypass arithmetic decoding node 216 to output a non-binary sao_offset value.

[0099] The SAO processing unit 204 performs an SAO process (offset process) on the decoded block output from the addition unit 203 to add an SAO offset value to a pixel value of each pixel included in the decoded block, which will be described in detail later.

[0100] The decoded block combining unit 205 combines the decoded blocks obtained in the SAO process to reconstruct the decoded image. In addition, the decoded block combining unit 205 stores the decoded image in the frame memory 206.

[0101] The frame memory 206 includes a memory device such as a random access memory (RAM) and a read-only memory (ROM).

[0102] The prediction unit 207 generates a prediction block by means of inter prediction or intra prediction. The prediction method to be applied by the prediction unit 207 is the same as that in the prediction unit 109.

[0103] <2-2 General Operations>

Next, operations of the image decoding device 200 will be described with reference to Fig. 13. Fig. 13 is a flow chart indicating a procedure in the image decoding device 200.

[0104] The SAO parameter decoding unit 210 in the image decoding device 200 performs variable length decoding on the SAO parameter (step S201).

[0105] The variable length decoding unit 201 in the image decoding device 200 performs variable length decoding on the binary signal stream to obtain quantized coefficients, and outputs the quantized coefficients to the inverse transform unit 202 (step S202).

[0106] The inverse transform unit 202 in the image decoding device 200 performs inverse quantization on the quantized coefficients to obtain frequency coefficients, and performs inverse transform on the obtained frequency coefficients to generate a difference block (step S203).

[0107] The prediction unit 207 in the image decoding device 200 generates a prediction block from the decoded image stored in the frame memory 206. In addition, the addition unit 203 in the image decoding device 200 adds the prediction block output from the prediction unit 207 with the difference block output from the inverse transform unit 202 to generate a decoded block (step S204).

[0108] The SAO processing unit 204 in the image decoding device 200 performs the SAO process (step S205). In the SAO process, the SAO processing unit 204 classifies each pixel of the decoded block into categories according to an SAO parameter and adds an offset value corresponding to each of the categories to the pixel. The SAO parameters include a sao_type_idx parameter indicating the category and a sao_offset parameter indicating the offset value.

[0109] The image decoding device 200 repeats the processes from Step 201 to Step 205 until decoding of all blocks included in the coded image is completed (step S206).

[0110] The decoded block combining unit 205 in the image decoding device 200 combines the decoded blocks to form a decoded image and stores the decoded image in the frame memory 206.

[0111] <2-2-1. SAO Parameter Variable Length Decoding Node Operations>

The operations of the SAO parameter decoding unit 210 according to Embodiment 2 will be described with reference to Fig. 14.

[0112] Fig. 14 is a flow chart indicating exemplary processes of the SAO parameter decoding unit 210 and describes in detail Step S205 of Fig. 13.

[0113] The sao_type_idx decoding unit 211 in the SAO parameter decoding unit 210 decodes the sao_type_idx (step S211).

[0114] The sao_offset decoding unit 212 in the SAO parameter decoding unit 210 decodes the sao_offset (step S212).

[0115] <2-2-2. Operations of node 212 decoding sao_offset>

The operations of the sao_offset decoding unit 212 according to Embodiment 2 will be described with reference to Fig. 15A. Fig. 15A is a flow chart indicating exemplary processes in the sao_offset decoding unit 212.

[0116] The arithmetic decoding switching unit 213 in the sao_offset decoding unit 212 determines the value of binIdx of the bin to be decoded (steps S221, S223 and S225).

[0117] When the value is 0 (Yes in step S221), the arithmetic decoding switching unit 213 outputs a bin to be decoded to the context 0 arithmetic decoding unit 214. The context 0 arithmetic decoding unit 214 performs context arithmetic decoding using context 0 (step S222).

[0118] When the value is 1 (No in step S221 and Yes in step S223), the arithmetic decoding switching unit 213 outputs a bin to be decoded to the context 1 arithmetic decoding unit 215. The context 1 arithmetic decoding unit 215 performs context arithmetic decoding using context 1 (step S224).

[0119] When the value is neither 0 nor 1 (NO in step S221 and NO in step S223), the arithmetic decoding switching unit 213 outputs the bin to be decoded to the bypass arithmetic decoding unit 216. The bypass arithmetic decoding unit 216 performs bypass arithmetic decoding.

[0120] The sao_offset decoding unit 212 repeats the processes from Step 221 to Step 226 until the value of the bin obtained in the arithmetic decoding is not 0, or the bin whose binIdx is 6 is decoded (step S226).

[0121] The sao_offset decoding unit 212 converts a bin string including the bins decoded by the sao_offset inverse transform unit 217 from binary form into a non-binary signal (step S227). The correspondence between the non-binary signals and the bin strings is the same as shown in Fig. 7 according to Embodiment 1.

[0122] <2-3. Advantages, etc.>

The image decoding device 200 according to Embodiment 2 can accelerate the processing by performing bypass arithmetic decoding on the bins in the second half of sao_offset.

[0123] <2-3-1. Varieties>

Although the bypass arithmetic decoding is performed on the bins after binIdx2 according to Embodiment 2, the processing is not limited to this. In the same manner as in Embodiment 1, the bypass arithmetic decoding can be performed on the bins after binIdx1 (Variation 1) and on all bins (Variation 2).

[0124] In Varieties 1 of Embodiment 2, context arithmetic decoding is performed on the bin at binIdx0, and bypass arithmetic decoding is performed on the bins after binIdx1. This process is a decoding process corresponding to the encoding process of Varieties 1 of Embodiment 1 of Fig. 8. Since the sao_offset decoding unit 212 of Fig. 12 of Varieties 1 performs bypass arithmetic decoding on the bins after binIdx1, it needs to include only the arithmetic decoding switching unit 213, the context 0 arithmetic decoding unit 214, the bypass arithmetic decoding unit 216, and the sao_offset inverse binary transform unit 217. In other words, the sao_offset decoding unit 212 according to Variation 1 of Embodiment 2 does not have to include the context 1 arithmetic decoding unit 215.

[0125] According to Variation 2 of Embodiment 2, the sao_offset decoding unit 212 performs bypass arithmetic decoding on each of the bins in binIdx. This process is a decoding process corresponding to the encoding process of Variation 2 of Embodiment 1 in Fig. 8.

[0126] Fig. 15B is a flow chart indicating exemplary processes (image decoding method) in the sao_offset decoding unit 212 according to Varieties 2 of Embodiment 2. In Fig. 15B, bypass arithmetic decoding is performed on all bins included in the sao_offset (steps S225 and S226 for performing the bypass arithmetic decoding), and the decoded sao_offset is converted back from binary form (step S227 for converting a bin string into an offset value). Since the bypass arithmetic decoding is performed on all bins, the processing can be the fastest in Varieties 2.

[0127] Since the sao_offset decoding unit 212 in Fig. 12 according to Varieties 2 in this document performs bypass arithmetic decoding on all bins, it needs to include only the bypass arithmetic decoding unit 216 and the sao_offset inverse conversion unit 217 from the binary form. In other words, the sao_offset decoding unit 212 according to Varieties 2 of Embodiment 2 does not need to include the arithmetic decoding switching unit 213, the context 0 arithmetic decoding unit 214, and the context 1 arithmetic decoding unit 215.

[0128] Although not illustrated, it is acceptable to perform context arithmetic decoding on binIdx 0 through 2 and bypass arithmetic decoding on binIdx after binIdx3, as in Embodiment 1.

[0129] (Other variations of Embodiments 1 and 2)

(1) Although Embodiments 1 and 2 (including Varieties) use sao_type_idx indicating a category of an SAO offset value and sao_offset indicating an SAO offset value as SAO parameters, the SAO parameters are not limited to this. The SAO parameters may include, for example, a parameter that should be auxiliary information for classifying pixels.

[0130] (2) In addition, although according to Embodiment 1 (including Varieties), the SAO parameter is encoded per block to be predicted, the encoding is not limited to this. The SAO parameter may not be encoded per block to be predicted, but rather per unit less than the block to be predicted, or vice versa per unit obtained by combining the blocks to be predicted. In addition, the SAO parameter is not encoded in the block to be predicted, but instead the value of another block may be copied and used.

[0131] Here, the image decoding device 200 according to Embodiment 2 (including Varieties) decodes the encoded SAO parameter according to the encoding size of the SAO parameter in the image encoding device 100.

[0132] (3) Although sao_offset takes values from 0 to 7 according to Embodiments 1 and 2 (including Varieties), sao_offset is not limited to this. It may take values of 8 or higher. In addition, sao_offset may take a negative value, in which case another parameter sao_offset_sign representing the sign bit may be provided.

[0133] (4) Furthermore, although Embodiment 1 (including Varieties) describes encoding of sao_offset, the processing is not limited thereto. The method for encoding the SAO parameter according to Embodiment 1 (including Varieties) can be applied to encoding other syntax that is to be added to the bit stream. Furthermore, when the method for encoding the SAO parameter according to Embodiment 1 and Varieties 1 thereof is applied to encoding syntaxes, and the use of the context (the number of bins and binIdx that use the context) is unified, the same variable-length encoding node is available when encoding syntaxes. Accordingly, the configuration of the device can be simplified.

[0134] Examples of syntaxes to which the method for encoding the SAO parameter according to Embodiment 1 and Variations 1 and 2 thereof is applied include ref_idx indicating a reference image index, merge_idx indicating a copy source of a motion vector, etc., and mpm_idx or intra_chroma_pred_mode indicating an intra prediction mode in accordance with the current HEVC standard (NPL 1).

[0135] Here, the image decoding device 200 according to Embodiment 2 (including Varieties) also decodes the syntax encoded using the method for encoding the SAO parameter using the method for decoding the SAO parameter.

[0136] (5) In addition, although the size of the block to be predicted according to Embodiments 1 and 2 (including Varieties) is a maximum of 32×32 pixels and a minimum of 4×4 pixels, the size is not limited to this. In addition, although Embodiments 1 and 2 (including Varieties) describe blocks to be predicted that have a variable size, the size of the blocks may be constant.

[0137] (6) In addition, the processing according to Embodiments 1 and 2 (including Varieties) can be implemented by software. The software can be distributed by, for example, downloading. In addition, the software can be distributed by recording it on recording media such as a CD-ROM. This is also true for the other Embodiments in the description.

[0138] (7) In Embodiments 1 and 2 (including Varieties), each of the functional blocks can be implemented in most cases by, for example, an MPU and a memory. In addition, processing by each of the functional blocks can usually be implemented by software (program), and such software is recorded on a recording medium such as a ROM. In addition, such software can be distributed by, for example, loading and recording on recording media such as a CD-ROM. Each of the functional blocks can be implemented by hardware (dedicated circuit).

[0139] (8) The processing described in Embodiments 1 and 2 (including Varieties) may be performed as centralized processing by a single device (system) or as decentralized processing by a plurality of devices. Here, a program may be executed by one or more computers. In other words, any of centralized processing and decentralized processing may be performed.

[0140] (9) Furthermore, each of the constituent elements according to Embodiments 1 and 2 (including Varieties) may be realized by specialized hardware or by executing a computer program suitable for the constituent element. Each of the constituent elements may be realized by a program execution unit, such as a central processing unit (CPU) and a processor that reads and executes a computer program recorded on a recording medium such as a hard disk or a semiconductor memory device.

[0141] In particular, each of the image encoding device 100 and the image decoding device 200 includes a control circuit and a storage device electrically connected to the control circuit (accessible from the control circuit). The control circuit includes at least one of specialized hardware and a program execution unit. When the control circuit includes a program execution unit, the storage device stores a computer program executed by the program execution unit.

[0142] The present invention is not limited to Embodiments 1 and 2 (including Variations), and various modifications are possible and included within the scope of the present invention.

[0143] (Implementation option 3)

An independent computer system can easily perform the processing described in each of Embodiments by recording a program for implementing the structure of the movie encoding method (image encoding method) or the movie decoding method (image decoding method) according to Embodiment on a recording medium. The recording medium may be any as long as the program can be recorded on it, such as a magnetic disk, an optical disk, a magneto-optical disk, an integrated circuit board, and a semiconductor memory device.

[0144] Applications of a movie encoding method (image encoding method) and a movie decoding method (image decoding method) according to each of the Embodiments, and a system using such applications will be described below. Features of the system include an image encoding device using the image encoding method, and an image encoding and decoding device including an image decoding device using the image decoding method. Other configurations of the system can be changed appropriately depending on the case.

[0145] Fig. 16 illustrates a general configuration of a content providing system ex100 for implementing content distribution services. An area for providing communication services is divided into cells of a desired size, and base stations ex106-ex110, which are fixed wireless stations, are located in each of the cells.

[0146] The content providing system ex100 is connected to devices such as a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a cellular telephone ex114, and a gaming device ex115 via the Internet ex101, an Internet service provider ex102, a telephone network ex104, and base stations ex106-ex110.

[0147] However, the configuration of the content providing system ex100 is not limited to the configuration shown in Fig. 16, and a combination in which any elements are connected is acceptable. In addition, each of the devices may be connected to the telephone network ex104 directly, rather than via the base stations ex106 to ex110, which are fixed wireless stations. In addition, the devices may be interconnected with each other by short-range wireless communication and others.

[0148] The camera ex113, such as a digital video camera, is capable of shooting moving images. The camera ex116, such as a digital video camera, is capable of shooting both still images and moving images. In addition, the cellular telephone ex114 may be a telephone that complies with any of standards such as Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), and High Speed Packet Access (HSPA). Alternatively, the cellular telephone ex114 may be a Personal Handheld Telephone System (PHS).

[0149] In the content providing system ex100, the streaming server ex103 is connected to the camera ex113 and others via the telephone network ex104 and the base station ex109, which enables distribution of a live broadcast and others. For such distribution, the content (for example, a video image of a live music broadcast) shot by a user using the camera ex113 is encoded as described above in each of Embodiments, and the encoded content is transmitted to the streaming server ex103. On the other hand, the streaming server ex103 streams distribution of the received content data to the clients upon their requests. The clients include a computer ex111, a PDA ex112, a camera ex113, a cellular phone ex114 and a gaming device ex115, which are capable of decoding the above-mentioned encoded data. Each of the devices that has received the distributed data decodes and reproduces the encoded data (that is, functions as an image decoding device according to an aspect of the present invention).

[0150] The captured data may be encoded by the camera ex113 or the streaming server ex103 that transmits the data, or the encoding processes may be distributed between the camera ex113 and the streaming server ex103. Similarly, the distributed data may be decoded by the clients or the streaming server ex103, or the decoding processes may be distributed between the clients and the streaming server ex103. Furthermore, the still image data and the moving image data captured not only by the camera ex113 but also by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The encoding processes may be performed by the camera ex116, the computer ex111, or the streaming server ex103, or distributed between them.

[0151] Furthermore, typically, the computer ex111 and the LSI ex500 included in each of the devices perform such encoding and decoding processes. The LSI ex500 may be configured from a single chip or a plurality of chips. Software for encoding and decoding a movie may be built into some type of recording medium (for example, a CD-ROM, a floppy disk, a hard disk) that is readable by the computer ex111 and others, and the encoding and decoding processes can be performed using this software. Furthermore, when the cellular phone ex114 is equipped with a camera, it is possible to transmit video data acquired by the camera. The video data is data encoded by the LSI ex500 included in the cellular phone ex114.

[0152] In addition, the streaming server ex103 may be composed of servers and computers and may decentralize data and process decentralized data, record or distribute data.

[0153] As described above, the clients can receive and reproduce the encoded data in the content providing system ex100. In other words, the clients can receive and decode the information transmitted by the user and reproduce the decoded data in real time in the content providing system ex100, so that the user who does not have any specific right and equipment can realize personal broadcasting.

[0154] The present invention is not limited to the above-mentioned content providing system ex100, and at least one of the movie encoding device (image encoding device) or the movie decoding device (image decoding device) described in each of Embodiments can be included in the digital broadcasting system ex200 shown in Fig. 17. More specifically, the broadcasting station ex201 communicates or transmits, via radio waves, to the broadcasting satellite ex202, multiplexed data obtained by multiplexing video data with audio data. The video data is data encoded according to the movie encoding method described in each of Embodiments (that is, data encoded by the image encoding device according to an aspect of the present invention). When receiving the video data, the broadcasting satellite ex202 transmits radio waves for broadcasting. Then, a consumer antenna ex204 capable of receiving satellite broadcasts receives the radio waves. A device such as a television ex300 (receiver) and a set-top box ex217 (STB) decodes the received multiplexed data and reproduces the data (that is, functions as an image decoding device according to an aspect of the present invention).

[0155] Furthermore, the reader/writer ex218 that (i) reads and decodes multiplexed data recorded on the recording medium ex215 such as a DVD and a BD, or (ii) encodes video signals on the recording medium ex215 and, in some cases, records data obtained by multiplexing an audio signal onto the encoded data, may include a movie decoding device or a movie encoding device, which are shown in each of Embodiments. In this case, the reproduced video signals are displayed on the monitor ex219, and another device or system can reproduce the video signals using the recording medium ex215 on which the multiplexed data is recorded. In addition, it is possible to implement a movie decoding device in a set-top box ex217 connected to a cable ex203 for cable television or to an antenna ex204 for satellite and/or terrestrial broadcasting, in order to display video signals on a monitor ex219 of a television ex300. The movie decoding device may be included in the television ex300 rather than in the set-top box.

[0156] Fig. 18 illustrates a television (receiver) ex300 that uses the movie encoding method and the movie decoding method described in each of Embodiments. The television ex300 includes: a tuner ex301 that receives or supplies multiplexed data obtained by multiplexing audio data and video data via an antenna ex204 or a cable ex203, etc. that receives a broadcast; a modulation/demodulation unit ex302 that demodulates the received multiplexed data or modulates the data into multiplexed data to supply it to the outside of the unit; and a multiplexing/demultiplexing unit ex303 that demultiplexes the modulated multiplexed data into video data and audio data or multiplexes the video data and audio data encoded by the signal processing unit ex306 into data.

[0157] Furthermore, the television ex300 further includes: a signal processing unit ex306 including an audio signal processing unit ex304 and a video signal processing unit ex305 that decodes audio data and video data and codes the audio data and video data, respectively (which function as an image coding device or an image decoding device according to an aspect of the present invention); a speaker ex307 that provides the decoded audio signal; and an output unit ex309 including a display unit ex308 that displays the decoded video signal, such as a display. Furthermore, the television ex300 includes an interface unit ex317 including an operation input unit ex312 that receives an input of a user action. Furthermore, the television ex300 includes a control unit ex310 that controls each constituent element of the television ex300 in general, and a power supply circuit ex311 that supplies power to each of the elements. In addition to the operation input unit ex312, the interface unit ex317 may include: a bridge ex313 that is connected to an external device such as a reader/writer ex218; a connector ex314 for enabling connection of a recording medium ex216 such as an SD card; a drive ex315 that is to be connected to an external recording medium such as a hard disk; and a modem ex316 that is to be connected to a telephone network. Here, the recording medium ex216 can electrically record information using a non-volatile/volatile semiconductor memory element for storage. The constituent elements of the television ex300 are connected to each other via a synchronous bus.

[0158] First, a configuration will be described in which the television ex300 decodes multiplexed data received from the outside via the antenna ex204 and the like and reproduces the decoded data. In the television ex300, as a result of receiving a user action from the remote control ex220 and the like, a multiplexing/demultiplexing unit ex303 demultiplexes the multiplexed data demodulated by the modulation/demodulation unit ex302, under the control of a control unit ex310 including a CPU. Furthermore, in the television ex300, an audio signal processing unit ex304 decodes the demultiplexed audio data, and a video signal processing unit ex305 decodes the demultiplexed video data, using the decoding method described in each of Embodiments. An output unit ex309 supplies the decoded video signal and the audio signal to the outside. When the output unit ex309 supplies a video signal and an audio signal, the signals can be temporarily stored in the buffers ex318 and ex319 and others so that the signals are reproduced in synchronism with each other. Furthermore, the television ex300 can read the multiplexed data not by broadcasting and the like, but from the recording media ex215 and ex216, such as a magnetic disk, an optical disk and an SD card. Next, a configuration will be described in which the television ex300 encodes an audio signal and a video signal and transmits the data to the outside or records the data on the recording medium. In the television ex300, as a result of receiving an operation of a user from a remote control ex220 and others, the audio signal processing unit ex304 encodes the audio signal, and the video signal processing unit ex305 encodes the video signal, under the control of the control unit ex310, using the encoding method that is described in each of the embodiments. The multiplexing/demultiplexing unit ex303 multiplexes the encoded video signal and the audio signal and supplies the resulting signal to the outside. When the multiplexing/demultiplexing unit ex303 multiplexes a video signal and an audio signal, the signals may be temporarily stored in the buffers ex320 and ex321 and others so that the signals are reproduced in synchronization with each other. Here, the buffers ex318 to ex321 may be multiple as illustrated, or at least one buffer may be shared in the television ex300. Furthermore, data may be stored in a buffer other than the buffers ex318 to ex321 so that overflow and underflow of the system can be avoided between the modulation/demodulation unit ex302 and the multiplexing/demultiplexing unit ex303, for example.

[0159] In addition, the television ex300 may include a configuration for receiving audio-visual input data from a microphone or a camera, in addition to a configuration for obtaining audio data and video data from a broadcast or a recording medium, and may encode the obtained data. Although in the description, the television ex300 may encode, multiplex, and provide data to the outside, it may not be capable of performing all the processes, but only capable of receiving, decoding, and providing data to the outside.

[0160] Furthermore, when the reader/writer ex218 reads or writes multiplexed data from or to the recording medium, one of the television ex300 and the reader/writer ex218 can decode or encode the multiplexed data, and the television ex300 and the reader/writer ex218 can distribute the decoding or encoding.

[0161] As an example, FIG. 19 illustrates a configuration of an information reproducing/recording unit ex400 when data is read from or written to an optical disc. The information reproducing/recording unit ex400 includes constituent elements ex401 to ex407 that will be described later. An optical pickup ex401 irradiates a laser spot onto a recording surface of a recording medium ex215 that is an optical disc to record information, and detects reflected light from the recording surface of the recording medium ex215 to read information. A modulation recording unit ex402 electrically controls a semiconductor laser included in the optical pickup ex401 and modulates the laser emission in accordance with the recorded data. A playback demodulation unit ex403 amplifies a playback signal obtained by electrically detecting reflected light from a recording surface using a photodetector included in the optical head ex401, and demodulates the playback signal by separating a signal component recorded on the recording medium ex215 to reproduce necessary information. A buffer ex404 temporarily stores information to be written on the recording medium ex215 and information reproduced from the recording medium ex215. A disc motor ex405 rotates the recording medium ex215. A servo control unit ex406 moves the optical head ex401 on a predetermined information track while controlling the rotation of the disc motor ex405 to follow a laser spot. A system control unit ex407 controls the information playback/recording unit ex400 as a whole. The reading and writing processes can be implemented by a system control unit ex407 that uses various information stored in a buffer ex404 and generates and adds new information as needed, and a modulation recording unit ex402, a playback demodulation unit ex403, and a servo control unit ex406 that record and reproduce information by means of an optical head ex401 while being controlled in a coordinated manner. The system control unit ex407 includes, for example, a microprocessor, and performs processing by causing a computer to execute a program for reading and writing.

[0162] Although the ex401 optical head is described as emitting a laser spot, it can perform high-density recording using near-field light.

[0163] Fig. 20 schematically illustrates a recording medium ex215 that is an optical disc. On the recording surface of the recording medium ex215, guide grooves are formed in a spiral, and an information track ex230 records in advance address information indicating an absolute position on the disc according to a change in the shape of the guide grooves. The address information includes information for specifying the positions of recording blocks ex231 that are a unit for recording data. A device that records and reproduces data reproduces the information track ex230 and reads the address information to determine the positions of the recording blocks. Furthermore, the recording medium ex215 includes a data recording area ex233, an inner circumference area ex232 and an outer circumference area ex234. The data recording area ex233 is an area for use in recording user data. An inner circumferential area ex232 and an outer circumferential area ex234 that are located inside and outside the data recording area ex233, respectively, are for special use other than recording user data. The information reproducing/recording unit ex400 reads and writes coded audio data, coded video data, or multiplexed data obtained by multiplexing coded audio data and coded video data from and onto the data recording area ex233 in the recording medium ex215.

[0164] Although the description describes an optical disc having a layer such as a DVD and a BD as an example, the optical disc is not limited to this and may be an optical disc having a multilayer structure and capable of recording on parts other than the surface. In addition, the optical disc may have a structure for multidimensional recording/reproduction, such as recording information using light with colors of different wavelengths in the same part of the optical disc and recording information having different layers from different angles.

[0165] Furthermore, in the digital broadcasting system ex200, the automobile ex210 having the antenna ex205 can receive data from the satellite ex202 and others and reproduce a video image on a display device such as the automobile navigation system ex211 installed in the automobile ex210. Here, the configuration of the automobile navigation system ex211 will be a configuration including, for example, a GPS receiving unit in the configuration illustrated in Fig. 18. The same will be true for the configuration of the computer ex111, the cellular phone ex114 and others.

[0166] Fig. 21A illustrates a cellular phone ex114 that uses the movie encoding method and the movie decoding method described in each of Embodiments. The cellular phone ex114 includes: an antenna ex350 for transmitting and receiving radio waves by a base station ex110; a camera ex365 capable of shooting moving and still images; and a display unit ex358 such as a liquid crystal display for displaying data such as a decoded video image shot by the camera ex365 or received by the antenna ex350. The cellular phone ex114 further includes: a body including a set of control buttons ex366; an audio output unit ex357 such as a speaker for outputting audio; an audio input unit ex356 such as a microphone for inputting audio; a storage device ex367 for storing captured video or still images, recorded audio, encoded or decoded data of received video, still images, emails or the like; and a connector ex364 that is an interface unit for a recording medium that stores data in the same manner as the storage device ex367.

[0167] Next, an example of a configuration of a cellular phone ex114 will be described with reference to Fig. 21B. In the cellular phone ex114, a main control unit ex360 configured to control as a whole each unit of a housing including a display unit ex358 and control buttons ex366 is mutually connected via a synchronous bus ex370 to a power supply circuit ex361, an operation input control unit ex362, a video signal processing unit ex355, a camera interface unit ex363, a liquid crystal display (LCD) driving unit ex359, a modulation/demodulation unit ex352, a multiplexing/demultiplexing unit ex353, an audio signal processing unit ex354, a connector ex364, and a memory ex367.

[0168] When a call end key or a power key is pressed as a result of a user operation, the power supply circuit ex361 supplies the corresponding nodes with power from the battery to activate the cellular telephone ex114.

[0169] In the cellular phone ex114, the audio signal processing unit ex354 converts the audio signals collected by the audio input unit ex356 in the speech conversion mode into digital audio signals under the control of the main control unit ex360 including a CPU, a ROM and a RAM. Then, the modulation/demodulation unit ex352 performs spread spectrum processing on the digital audio signals, and the transmission and reception unit ex351 performs digital-to-analog conversion and frequency conversion on the data to transmit the resulting data via the antenna ex350. Then, the modulation/demodulation unit ex352 performs inverse spread spectrum processing on the data, and the audio signal processing unit ex354 converts them into analog audio signals to output them via the audio output unit ex357.

[0170] Furthermore, when an e-mail is transmitted in the data communication mode, text data of the e-mail inputted by operating the operation buttons ex366 and others in the body is sent to the master control unit ex360 via the operation input control unit ex362. The master control unit ex360 causes the modulation/demodulation unit ex352 to perform spread spectrum processing on the text data, and the transmission and reception unit ex351 performs digital-to-analog conversion and frequency conversion on the resulting data, so as to transmit the data to the base station ex110 via the antenna ex350. When an e-mail is received, processing that is approximately the inverse of the processing for transmitting the e-mail is performed on the received data, and the resulting data is provided to the display unit ex358.

[0171] When a video image, still images, or a video image and audio are transmitted in a data transmission mode, the video signal processing unit ex355 (functioning as an image encoding device according to an aspect of the present invention) compresses and encodes video signals input from the camera ex365 using the movie encoding method shown in each of Embodiments, and transmits the encoded video data to the multiplexing/demultiplexing unit ex353. In contrast, when the camera ex365 shoots video images, still images, and others, the audio signal processing unit ex354 encodes audio signals collected by the audio input unit ex356 and transmits the encoded audio data to the multiplexing/demultiplexing unit ex353.

[0172] The multiplexing/demultiplexing unit ex353 multiplexes the encoded video data supplied from the video signal processing unit ex355 and the encoded audio data supplied from the audio signal processing unit ex354 using a predetermined method. Then, the modulation/demodulation unit ex352 performs spread spectrum processing on the multiplexed data, and the transmission and reception unit ex351 performs digital-to-analog conversion and frequency conversion on the data to transmit the resulting data via the antenna ex350.

[0173] When receiving data of a video file that is linked to a web page and others in a data transfer mode, or when receiving an e-mail with an attached video image and/or audio, in order to decode the multiplexed data received via the antenna ex350, the multiplexing/demultiplexing unit ex353 demultiplexes the multiplexed data into a video data bitstream and an audio data bitstream, and supplies the video signal processing unit ex355 with the encoded video data and the audio signal processing unit ex354 with the encoded audio data via the synchronous bus ex370. The video signal processing unit ex355 (functioning as an image decoding device according to an aspect of the present invention) decodes a video signal using a movie decoding method corresponding to the movie encoding method shown in each of Embodiments, and then the display unit ex358 displays, for example, a video image and still images included in a video file linked to a web page, via the LCD control unit ex359. Furthermore, the audio signal processing unit ex354 decodes an audio signal, and the audio output unit ex357 provides audio.

[0174] Furthermore, similar to the television ex300, a terminal such as the cellular telephone ex114 may have 3 types of configurations including not only (i) a transmission and a reception terminal including an encoding device and a decoding device, but also (ii) a transmission terminal including only an encoding device, and (iii) a reception terminal including only a decoding device. Although in the description, the digital broadcasting system ex200 receives and transmits multiplexed data obtained by multiplexing video data with audio data, the multiplexed data may be data obtained by multiplexing video data with not audio data but character data related to a video image, and may not be multiplexed data but video data itself.

[0175] In essence, the method for encoding a movie or the method for decoding a movie in each of the Embodiments can be used on any of the described devices and systems. Thus, the advantages described in each of the Embodiments can be obtained.

[0176] Furthermore, the present invention is not limited to the Embodiments, and various modifications and changes are possible without departing from the scope of the present invention.

[0177] (Implementation option 4)

The video data may be generated by switching, as necessary, between (i) the movie encoding method or movie encoding device shown in each of the Embodiments and (ii) the movie encoding method or movie encoding device according to another standard, such as MPEG-2, MPEG4-AVC and VC-1.

[0178] Here, when a plurality of video data that conforms to different standards is generated and then decoded, it is necessary to select decoding methods that conform to different standards. However, since it is impossible to determine which standard each of the plurality of video data to be decoded conforms to, there is a problem that an appropriate decoding method cannot be selected.

[0179] In order to solve this problem, multiplexed data obtained by multiplexing video data with audio data and others has a structure including identification information indicating which standard the video data conforms to. A specific structure of multiplexed data including video data generated by a movie encoding method or by a movie encoding device shown in each of Embodiments will be described below. The multiplexed data is a digital stream in the MPEG-2 transport stream format.

[0180] Fig. 22 illustrates the structure of multiplexed data. As illustrated in Fig. 22, the multiplexed data can be obtained by multiplexing at least one of a video stream, an audio stream, a display graphics (PG) stream, and an interactive graphics stream. The video stream represents a primary video image and a secondary video image of a movie, the audio stream represents a primary audio portion and a secondary audio portion to be mixed with the primary audio portion, and the display graphics stream represents subtitles of the movie. Here, the primary video image is a normal video image shown on the screen, and the secondary video image is a video image to be shown in a smaller window in the main video image. In addition, the interactive graphics stream represents an interactive screen to be formed by placing GUI components on the screen. The video stream is encoded by a movie encoding method or by a movie encoding device shown in each of the Embodiments, or by a movie encoding method or by a movie encoding device in accordance with a conventional standard, such as MPEG-2, MPEG4-AVC and VC-1. The audio stream is encoded in accordance with a standard, such as Dolby-AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD and linear PCM.

[0181] Each stream included in the multiplexed data is identified by a PID. For example, 0×1011 is assigned to a video stream to be used for the movie video, 0×1100-0×111F are assigned to audio streams, 0×1200-0×121F are assigned to demonstration graphics streams, 0×1400-0×141F are assigned to interactive graphics streams, 0×1B00-0×1B1F are assigned to video streams to be used for the secondary video of the movie, and 0×1A00-0×1A1F are assigned to audio streams to be used for the secondary video to be mixed with the primary audio.

[0182] Fig. 23 schematically illustrates how data is multiplexed. First, a video stream ex235 composed of video frames and an audio stream ex238 composed of audio frames are converted into a stream of PES packets ex236 and a stream of PES packets ex239, and further into TS packets ex237 and TS packets ex240, respectively. Similarly, data from a demonstration graphic stream ex241 and data from an interactive graphic stream ex244 are converted into a stream of PES packets ex242 and a stream of PES packets ex245, and further into TS packets ex243 and TS packets ex246, respectively. These TS packets are multiplexed into a stream to obtain multiplexed data ex247.

[0183] Fig. 24 illustrates in more detail how a video stream is stored in a PES packet stream. The first stripe in Fig. 24 shows a stream of video frames in the video stream. The second stripe shows a stream of PES packets. As indicated by the arrows labeled yy1, yy2, yy3 and yy4 in Fig. 24, the video stream is divided into pictures in the form of I-pictures, B-pictures and P-pictures, each of which is a unit of video presentation, and the pictures are stored in the payload of each of the PES packets. Each of the PES packets has a PES header, and the PES header stores a Playback Timestamp (PTS) indicating the display time of the picture and a Decoding Timestamp (DTS) indicating the decoding time of the picture.

[0184] Fig. 25 illustrates the format of the TS packets that are ultimately to be written into the multiplexed data. Each of the TS packets is a 188-byte fixed-length packet including a 4-byte TS header containing information such as a PID for identifying a stream and a 184-byte TS payload for storing data. PES packets are separated and stored in the TS payloads. When a BD ROM is used, each of the TS packets is given a 4-byte TP_Extra_Header, respectively resulting in 192-byte source packets. The source packets are written into the multiplexed data. The TP_Extra_Header stores information such as Arrival_Time_Stamp (ATS). The ATS indicates the transmission start time at which each of the TS packets should be passed to the PID filter. The numbers increasing from the start of the multiplexed data are called source packet numbers (SPNs), as shown in the lower part of Fig. 25.

[0185] Each of the TS packets included in the multiplexed data includes not only audio, video, subtitle and other streams, but also a Program Association Table (PAT), a Program Mapping Table (PMT) and a Program Reference Signal (PCR). The PAT indicates what the PID in the PMT used in the multiplexed data indicates, and the PID of the PAT itself is registered as zero. The PMT stores the PIDs of the video, audio, subtitle and other streams included in the multiplexed data, and information about the attributes of the streams corresponding to the PIDs. The PMT also contains various descriptors related to the multiplexed data. The descriptors have information such as copy control information indicating whether copying of the multiplexed data is permitted. PCR stores STC timing information corresponding to ATS indicating when the PCR packet is transmitted to the decoder in order to achieve synchronization between the Arrival Timer (ATC), which is the time axis of ATS, and the System Timer (STC), which is the time axis of PTS and DTS.

[0186] Fig. 26 illustrates the data structure of the PMT in detail. A PMT header is located at the beginning of the PMT. The PMT header describes the length of the data included in the PMT, etc. After the PMT header, there are a plurality of descriptors related to the multiplexed data. The descriptors describe information such as copy control information. After the descriptors, there are a plurality of pieces of stream information related to the streams included in the multiplexed data. Each piece of stream information includes stream descriptors that describe information such as a stream type for identifying a stream compression codec, a stream PID, and stream attribute information (such as a frame rate or an aspect ratio). The stream descriptors are equal in number to the number of streams in the multiplexed data.

[0187] When multiplexed data is recorded on a recording medium etc., it is recorded together with multiplexed data information files.

[0188] Each of the multiplexed data information files is control information about the multiplexed data, as shown in Fig. 27. The multiplexed data information files are in one-to-one correspondence with the multiplexed data, and each of the files includes information about the multiplexed data, information about stream attributes, and an input map.

[0189] As illustrated in Fig. 27, the information about the multiplexed data includes a system rate, a playback start time, and a playback end time. The system rate indicates the maximum transmission rate at which the final decoder of the system, which will be described later, transmits the multiplexed data to the PID filter. The intervals of the ATS included in the multiplexed data are set to be no higher than the system rate. The playback start time indicates a PTS in a video frame at the beginning of the multiplexed data. An interval of one frame is added to the PTS in a video frame at the end of the multiplexed data, and the PTS is set at the playback end time.

[0190] As shown in Fig. 28, a piece of attribute information is registered in the stream attribute information for each PID of each stream included in the multiplexed data. Each piece of attribute information contains different information depending on whether the corresponding stream is a video stream, an audio stream, a display graphics stream, or an interactive graphics stream. Each piece of video stream attribute information carries information including what kind of compression codec is used to compress the video stream, and a resolution, an aspect ratio, and a frame rate of pieces of image data that are included in the video stream. Each piece of audio stream attribute information carries information including what kind of compression codec is used to compress the audio stream, how many channels are included in the audio stream, what language the audio stream supports, and what the sampling frequency is. The video stream attribute information and the audio stream attribute information are used to initialize the decoder before the player reproduces the information.

[0191] In Embodiment 4, the multiplexed data to be used belongs to a stream type included in the PMT. Furthermore, when the multiplexed data is recorded on the recording medium, video stream attribute information included in the multiplexed data information is used. More specifically, the movie encoding method or the movie encoding device described in each of Embodiments includes a step or a unit for assigning unique information indicating video data generated by the movie encoding method or the movie encoding device in each of Embodiments to a stream type included in the PMT or the video stream attribute information. With this structure, the video data generated by the movie encoding method or the movie encoding device described in each of Embodiments can be distinguished from video data that conforms to another standard.

[0192] Furthermore, Fig. 29 illustrates the steps of the movie decoding method according to Embodiment 4. In step exS100, a stream type included in the PMT or attribute information of the video stream is obtained from the multiplexed data. Next, in step exS101, it is determined whether the stream type or the attribute information of the video stream indicates that the multiplexed data is generated by the movie encoding method or by the movie encoding device in each of Embodiments. When it is determined that the stream type or the attribute information of the video stream indicates that the multiplexed data is generated by the movie encoding method or by the movie encoding device in each of Embodiments, then in step exS102, the stream type or the attribute information of the video stream is decoded by the movie decoding method in each of Embodiments. In addition, when the stream type or the attribute information of the video stream indicates conformity with conventional standards such as MPEG-2, MPEG4-AVC and VC-1, in step exS103, the stream type or the attribute information of the video stream is decoded according to the movie decoding method in accordance with conventional standards.

[0193] As such, assigning a new unique value to the stream type or the video stream attribute information makes it possible to determine whether the movie decoding method or the movie decoding device described in each of Embodiments can perform decoding. Even when multiplexed data that conforms to a different standard is input, an appropriate decoding method or device can be selected. In this way, it becomes possible to decode the information without any error. Furthermore, the movie encoding method or device or the movie decoding method or device in Embodiment 4 can be used in the devices and systems described above.

[0194] (Implementation option 5)

Each of the movie encoding method, the movie encoding device, the movie decoding method and the movie decoding device in each of the embodiments is usually implemented as an integrated circuit or a Large Scale Integrated Circuit (LSI). As an example of an LSI, Fig. 30 illustrates a configuration of an LSI ex500 that is made into a single chip. The LSI ex500 includes elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508 and ex509 that will be described later, and these elements are connected to each other by a bus ex510. The power supply circuit ex505 is activated by supplying power to each of the elements when the power supply circuit ex505 is turned on.

[0195] For example, when encoding is performed, the LSI ex500 receives an audio-visual signal from the microphone ex117, the camera ex113, and others through the audio-visual input/output ex509 under the control of the control unit ex501 including the CPU ex502, the memory controller ex503, the stream controller ex504, and the driving frequency adjustment unit ex512. The received audio-visual signal is temporarily stored in an external memory ex511 such as an SDRAM. Under the control of the control unit ex501, the stored data is segmented into data portions according to the volume and processing speed to be supplied to the signal processing unit ex507. Then, the signal processing unit ex507 encodes the audio signal and/or the video signal. Here, the encoding of the video signal is the encoding described in each of Embodiments. Furthermore, the signal processing unit ex507 sometimes multiplexes the encoded audio data and the encoded video data, and the stream input/output ex506 supplies the multiplexed data to the outside. The provided multiplexed data is transmitted to the base station ex107 or recorded on the recording media ex215. When sets of data are multiplexed, the sets of data should be temporarily stored in the buffer ex508 so that the sets of data are synchronized with each other.

[0196] Although the memory device ex511 is an element outside the LSI ex500, it can be included in the LSI ex500. The buffer ex508 is not limited to one buffer, but can be composed of buffers. In addition, the LSI ex500 can be made into one chip or a plurality of chips.

[0197] Furthermore, although the control unit ex501 includes a CPU ex502, a memory controller ex503, a flow controller ex504, a driving frequency adjustment unit ex512, the configuration of the control unit ex501 is not limited to this. For example, the signal processing unit ex507 may further include a CPU. Including another CPU in the signal processing unit ex507 can improve the processing speed. Furthermore, as another example, the CPU ex502 may include a signal processing unit ex507 or an audio signal processing unit that is a part of the signal processing unit ex507. In such a case, the control unit ex501 includes the signal processing unit ex507 or a CPU ex502 including a part of the signal processing unit ex507.

[0198] The name used here is LSI, but it may also be called system LSI, super-LSI, or ultra-LSI depending on the degree of integration.

[0199] In addition, the methods for achieving integration are not limited to the LSI, and a dedicated circuit or a general-purpose processor, etc. can also achieve integration. For the same purpose, a field-programmable gate array (FPGA) that can be programmed after the production of the LSI, or a reconfigurable processor that allows reconfiguration of the connection or configuration of the LSI can be used. The programmable logic device can generally perform the movie encoding method and the movie decoding method according to the Embodiments and Variations by loading or reading from a memory a program included in software or firmware.

[0200] In the future, with the development of semiconductor technology, a completely new technology may replace LSI. Functional blocks can be integrated using such technology. There is a possibility that the present invention is applied to biotechnology.

[0201] (Implementation option 6)

When video data generated by the motion picture encoding method or by the motion picture encoding device described in each of Embodiments is decoded, compared with the case of decoding video data that conforms to a conventional standard such as MPEG-2, MPEG4-AVC, and VC-1, the amount of calculation is likely to increase. Therefore, the LSI ex500 needs to be set to a drive frequency higher than that of the CPU ex502 to be used when video data conforming to the conventional standard is decoded. However, when the drive frequency is set higher, there is a problem that power consumption increases.

[0202] In order to solve this problem, a movie decoding device, such as a television ex300 and an LSI ex500, is configured to determine which standard the video data conforms to and switch between driving frequencies according to the determined standard. Fig. 31 illustrates a configuration of ex800 in Embodiment 6. A driving frequency switching unit ex803 sets a driving frequency to a higher driving frequency when the video data is generated by the movie encoding method or by the movie encoding device described in each of Embodiments. Then, the driving frequency switching unit ex803 instructs a decoding processing unit ex801 that executes the movie decoding method described in each of Embodiments to decode the video data. When the video data conforms to a conventional standard, the driving frequency switching unit ex803 sets a driving frequency to a lower driving frequency than that of the video data generated by the movie encoding method or by the movie encoding device described in each of Embodiments. Then, the driving frequency switching unit ex803 instructs the decoding processing unit ex802 that conforms to the conventional standard to decode the video data.

[0203] More specifically, the driving frequency switching unit ex803 includes a CPU ex502 and a driving frequency adjustment unit ex512 in Fig. 30. Here, each of the decoding processing unit ex801 that performs the movie decoding method described in each of Embodiments and the decoding processing unit ex802 that conforms to a conventional standard corresponds to the signal processing unit ex507 in Fig. 30. The CPU ex502 determines which standard the video data conforms to. Then, the driving frequency adjustment unit ex512 determines the driving frequency based on the signal from the CPU ex502. Furthermore, the signal processing unit ex507 decodes the video data based on the signal from the CPU ex502. For example, the identification information described in Embodiment 4 is likely to be used to identify the video data. The identification information is not limited to that described in Embodiment 4, but may be any information as long as the information indicates what standard the video data conforms to. For example, when it is possible to determine what standard the video data conforms to based on an external signal for determining that the video data is used for television or a disc, etc., the determination can be made based on such an external signal. Furthermore, the CPU ex502 selects a driving frequency based on, for example, a look-up table in which the standards of the video data are associated with driving frequencies, as shown in Fig. 33. The driving frequency can be selected by storing the look-up table in the buffer ex508 and the internal memory LSI, and by referring to the look-up table by the CPU ex502.

[0204] FIG. 32 illustrates steps for executing the method in Embodiment 6. First, in step exS200, the signal processing unit ex507 obtains identification information from the multiplexed data. Next, in step exS201, the CPU ex502 determines whether video data is generated based on the identification information by the encoding method and the encoding device described in each of Embodiments. When the video data is generated by the encoding method or the encoding device described in each of Embodiments, in step exS202, the CPU ex502 supplies to the driving frequency adjustment unit ex512 a signal for setting the driving frequency to a higher driving frequency. Then, the driving frequency adjustment unit ex512 sets the driving frequency to the higher driving frequency. On the other hand, when the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG4-AVC and VC-1, in step exS203, the CPU ex502 transmits to the driving frequency adjustment unit ex512 a signal for setting the driving frequency to a lower driving frequency. Then, the driving frequency adjustment unit ex512 sets the driving frequency to a lower driving frequency than in the case where the video data is generated by the encoding method or encoding device described in each of Embodiments.

[0205] In addition, together with switching the excitation frequencies, it is possible to enhance the power saving effect by changing the voltage to be supplied to the LSI ex500 or the device including the LSI ex500. For example, when the excitation frequency is set lower, the voltage to be supplied to the LSI ex500 or the device including the LSI ex500 is likely to be set to a lower voltage than in the case where the excitation frequency is set higher.

[0206] Furthermore, as a method for setting the driving frequency, when the amount of calculation for decoding is larger, the driving frequency can be set higher, and when the amount of calculation for decoding is smaller, the driving frequency can be set lower. Thus, the setting method is not limited to the methods described above. For example, when the amount of calculation for decoding video data according to MPEG4-AVC is larger than the amount of calculation for decoding video data generated by the movie encoding method or by the movie encoding device described in each of Embodiments, the driving frequency is likely to be set in the reverse order of the setting described above.

[0207] Furthermore, the method for setting the driving frequency is not limited to the method for setting the driving frequency lower. For example, when the identification information indicates that the video data is generated by the movie encoding method or by the movie encoding device described in each of Embodiments, the voltage to be supplied to the LSI ex500 or the device including the LSI ex500 is likely to be set higher. When the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG4-AVC and VC-1, the voltage to be supplied to the LSI ex500 or the device including the LSI ex500 is likely to be set lower. As another example, when the identification information indicates that the video data is generated by the movie encoding method or by the video encoding device described in each of Embodiments, the driving of the CPU ex502 is likely not to be suspended. When the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG4-AVC, and VC-1, the driving of the CPU ex502 is likely to be suspended at a predetermined time because the CPU ex502 has excess processing capacity. Even when the identification information indicates that the video data is generated by the movie encoding method or by the movie encoding device described in each of Embodiments, in a case where the CPU ex502 has excess processing capacity, the driving of the CPU ex502 is likely to be suspended at a predetermined time. In such a case, the suspension time is likely to be set shorter than in a case where the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG4-AVC, and VC-1.

[0208] Accordingly, it is possible to enhance the power saving effect by switching between driving frequencies in accordance with the standard to which the video data conforms. In addition, when the LSI ex500 or a device including the LSI ex500 is driven using a battery, the battery life can be extended along with the power saving effect.

[0209](Implementation option 7)

There are cases where a plurality of video data that conform to different standards is provided to devices and systems such as a television and a cellular phone. In order to be able to decode a plurality of video data that conform to different standards, the signal processing unit ex507 of the LSI ex500 must conform to different standards. However, there are problems of increasing the scale of the circuit of the LSI ex500 and increasing the cost when the signal processing units ex507 that conform to the respective standards are used separately.

[0210] In order to solve these problems, a configuration is provided in which a decoding processing unit for implementing the movie decoding method described in each of Embodiments and a decoding processing unit that conforms to a conventional standard such as MPEG-2, MPEG4-AVC and VC-1 are partially shared. Ex900 in FIG. 34A shows an example of this configuration. For example, the movie decoding method described in each of Embodiments and the movie decoding method that conforms to MPEG4-AVC have partially common processing details such as entropy coding, inverse quantization, deblocking filtering and motion compensation. The processing details to be shared are likely to include using the decoding processing unit ex902 that conforms to MPEG4-AVC. In contrast, the dedicated decoding processing unit ex901 may be used for other processing that is unique to the present invention and does not conform to MPEG-4 AVC. A decoding processing unit for implementing the movie decoding method described in each of the Embodiments may be shared for processing to be distributed, and a dedicated decoding processing unit may be used for processing unique to the processing in MPEG4-AVC.

[0211] Furthermore, ex1000 in Fig. 34B shows another example in which the processing is partially distributed. This example uses a configuration including a dedicated decoding processing unit ex1001 that supports processing unique to an aspect of the present invention, a dedicated decoding processing unit ex1002 that supports processing unique to another conventional standard, and a decoding processing unit ex1003 that supports processing to be distributed between the movie decoding method of the aspect of the present invention and the conventional movie decoding method. Here, the dedicated decoding processing units ex1001 and ex1002 are not necessarily dedicated to the processing of the aspect of the present invention and the processing of the conventional standard, but may be units capable of implementing common processing. Furthermore, the configuration of Embodiment 7 can be implemented using the LSI ex500.

[0212] In essence, the scale reduction of the LSI circuit and the cost reduction are possible by sharing the decoding processing unit for the processing that needs to be distributed between the movie decoding method of the present invention and the movie decoding method according to the conventional standard.

[Industrial applicability]

[0213] The present invention has the advantage of accelerating processing without reducing encoding efficiency and is applicable to various uses such as storing, transmitting and exchanging data. The present invention is applicable, for example, to information display devices and image capturing devices such as televisions, digital video recording devices, automobile navigation systems, mobile phones, digital still cameras and digital video cameras, and is very suitable for practical use.

[List of positional designations]

[0214] 100 Image coding device

101 Blocking Node

102 Subtraction Node

103 Conversion Node

104 Variable Length Coding Node

105, 202 Inverse Transformation Node

106, 203 Folding Knot

107, 204 SAO processing unit

108, 206 Frame memory

109, 207 Node of Prediction

110 SAO Parameter Coding Node

111 Coding node sao_type_idx

112 Coding node sao_offset

113 Node convert sao_offset to binary form

114 Arithmetic coding switching unit

115 Context Arithmetic Coding Node 0

116 Context Arithmetic Coding Node 1

117 Bypass arithmetic coding node

200 Image decoding device

201 Variable Length Decoding Node

205 Node for combining decoded blocks

210 SAO Parameter Decoding Node

211 Decoding node sao_type_idx

212 Decoding node sao_offset

213 Arithmetic Decoding Switching Node

214 Context Arithmetic Decoding Node 0

215 Context Arithmetic Decoding Node 1

216 Bypass arithmetic decoding node

217 Inverse conversion node sao_offset from binary form

Claims (10)

A decoding method for decoding a block with a sample adaptive offset (SAO) value using arithmetic decoding from a bit stream, wherein the decoding method comprises the steps of: decode the quantized block parameter (201); inverse quantization and inverse transformation of the quantized parameter are performed to form the reconstructed residual block (202); forming a reconstructed block by adding the reconstructed residual block and the predicted block (203); decode the SAO type parameter from the bitstream using constant probability bypass arithmetic decoding, wherein the SAO type parameter indicates whether the SAO value is a boundary offset value or a range offset value, and the SAO value is to be added to the pixel value of the reconstructed image; decoding a plurality of bins from the bitstream using constant probability bypass arithmetic decoding, wherein the plurality of bins represents an integer indicating the magnitude of the SAO value; determining an integer indicating the magnitude of the SAO value based on a plurality of bins decoded using constant probability bypass arithmetic coding; and determine the SAO value using the decoded SAO type parameter and the specified integer; add the decoded block to the said determined SAO value (204).