JP2006014086A - Moving image encoding apparatus and moving image encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress image quality deterioration in an inter-frame when encoding a moving image using motion prediction. <P>SOLUTION: An encoding section (206) for encoding a moving image using inter-frame motion prediction includes: a a dividing part (302) for dividing each frame into a plurality of divided areas; an ROI tile determining part (317) for determining an important area out of the frames; inter-frame predicting means (310, 314) for searching highly correlated pixel sets for each divided area of the frame as an encoding target within the range of the important area of the preceding frame, and outputting differential data while taking a difference between data in the divided area and data of the searched pixel sets; and encoding means (303-308) for encoding the differential data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving image encoding apparatus and method, and more particularly, to a moving image encoding apparatus and method for encoding a moving image using motion prediction.

  In recent years, contents flowing through a network have become larger and diversified from character information to still image information and further to moving image information. Along with this, the development of an encoding technique for compressing the information amount has progressed, and the developed encoding technique has been widely spread by international standardization.

  On the other hand, the network itself has been increased in capacity and diversified, and one content has passed through various environments before it reaches the receiving side from the transmitting side. In addition, the processing performance of transmission / reception side devices is diversified. While PCs that are mainly used as transmission / reception devices have greatly improved performance such as CPU performance and graphics performance, various devices such as PDAs, mobile phones, TVs, and hard disk recorders have network connection functions. It has become to. For this reason, attention has been paid to a function called scalability that can cope with changing communication line capacity and processing performance of the receiving side device with a single data.

  The JPEG 2000 encoding method is widely known as a still image encoding method having this scalability function. This method is internationally standardized and is described in detail in Non-Patent Document 1. The feature is that the input image data is subjected to a discrete wavelet transform (DWT) and separated into a plurality of frequency bands. These coefficients are quantized and the values are arithmetically encoded for each bit plane. By encoding and decoding as many bit planes as necessary, it is possible to control a fine hierarchy.

  In addition, the JPEG2000 encoding method also realizes a technology called ROI (Region Of Interest) that relatively improves the image quality of a region of interest in an image, which is not found in the conventional encoding technology.

  FIG. 23 shows an encoding unit according to the JPEG2000 encoding method. A tile dividing unit 9001 divides an input image into a plurality of areas (tiles). This feature is optional. The DWT unit 9002 performs discrete wavelet transform and separates it into frequency bands. The quantization unit 9003 quantizes each coefficient. The ROI designation unit 9007 is an option, and an area of interest can be set. The quantization unit 9003 performs upshifting. The entropy encoding unit 9004 performs entropy encoding using an EBCOT (Embedded Block Coding with Optimized Truncation) method, and the bit truncation unit 9005 truncates the lower bits as necessary to perform rate control. The code forming unit 9006 adds header information, selects various scalability functions, and outputs encoded data.

  FIG. 24 shows a decoding unit according to the JPEG2000 encoding method. The code analysis unit 9020 analyzes the header and obtains information for configuring a hierarchy. The bit truncation unit 9021 truncates the lower order bits of the input encoded data corresponding to the capacity of the internal buffer and the decoding processing capability. The entropy decoding unit 9022 decodes the encoded data of the EBCOT encoding method, and obtains quantized wavelet transform coefficients. The inverse quantization 9023 unit performs inverse quantization on this, and the inverse DWT unit performs inverse discrete wavelet transform to reproduce image data. A tile combining unit 9025 combines a plurality of tiles and reproduces image data.

  The Motion JPEG2000 system that performs moving picture coding by making this JPEG2000 coding system correspond to each frame of a moving picture is also recommended (see, for example, Non-Patent Document 2). In this method, encoding processing is performed independently for each frame, and encoding is not performed using time correlation, so that redundancy remains between frames. For this reason, there is a problem that it is difficult to effectively reduce the code amount as compared with the moving picture coding method using time correlation.

  On the other hand, in the MPEG encoding method, motion compensation is performed to improve encoding efficiency (see, for example, Non-Patent Document 3). FIG. 25 shows the configuration of the encoding unit. The block division unit 9031 divides the block into 8 × 8 blocks, the difference unit 9032 subtracts the motion compensation prediction data, the DCT unit 9033 performs discrete cosine transform, and the quantization unit 9034 performs quantization. The result is encoded by the entropy encoding unit 9035, the header information is added by the code forming unit 9036, and the encoded data is output.

  At the same time, the inverse quantization unit 9037 performs inverse quantization at the same timing as the processing of the entropy encoding unit 9035, the inverse DCT unit 9038 performs inverse transform of the discrete cosine transform, the addition unit 9039 adds the prediction data, and the frame memory 9040 To store. The motion compensation unit 9041 obtains a motion vector with reference to the input image and the reference frame stored in the frame memory 9040, and generates prediction data.

ISO / IEC15444-1 (Information technology-JPEG 2000 image coding system-Part 1: Core coding system) ISO / IEC 15444-3 (Information technology-JPEG 2000 image coding system Part 3: Motion JPEG 2000) “Latest MPEG Textbook”, 76 pages, etc. ASCII Publishing Bureau 1994

  For the purpose of improving the efficiency of JPEG2000 encoding, there is a compression method in which motion compensation is added to JPEG2000. In such a moving image compression method, as shown in FIG. 26, when the prediction destination data is partially discarded due to truncation of lower bit planes, prediction errors accumulate, and the image quality of the inter frame increases. There was a problem of deterioration.

  The present invention has been made in view of the above problems, and an object thereof is to suppress image quality deterioration in an inter frame when a moving image is encoded using motion prediction.

  In order to achieve the above object, a moving picture coding apparatus according to the present invention for coding a moving picture using inter-frame motion prediction includes dividing means for dividing each frame into a plurality of divided areas, and an important area from within the frame. A pixel unit having a high correlation for each divided region of the frame to be encoded within the important region of the previous frame, and data of each divided region and the searched pixel set Inter-frame prediction means for taking the difference from the data and outputting the difference data, and encoding means for encoding the difference data.

  The moving image encoding method of the present invention for encoding a moving image using inter-frame motion prediction includes a dividing step of dividing each frame into a plurality of divided regions, and a determining step of determining an important region from within the frame. In the range of the important area of the previous frame, a pixel set having high correlation is searched for each divided area of the encoding target frame, and the difference between the data of each divided area and the searched pixel set data is calculated. An inter-frame prediction step for outputting difference data, and an encoding step for encoding the difference data.

  According to another configuration, the moving image encoding apparatus of the present invention that encodes a moving image using inter-frame motion prediction includes a dividing unit that divides each frame into a plurality of divided regions, Each divided region of the encoding target frame is determined from a determining unit that determines a region, a conversion unit that performs data conversion for each divided region and generates a conversion coefficient, and a conversion coefficient corresponding to the range of the important region of the previous frame. An inter-frame prediction unit that searches for a transform coefficient having a high correlation for each transform coefficient, takes a difference between the transform coefficient of each divided region and the found transform coefficient, and outputs difference data, and encodes the difference data And encoding means for converting.

  In addition, the moving image encoding method of the present invention that encodes a moving image for each frame using inter-frame motion prediction determines a division step for dividing each frame into a plurality of divided regions and an important region from within the frame. From the determination step, the conversion step of performing data conversion for each divided region, and generating the conversion coefficient, and the conversion coefficient corresponding to the range of the important region of the previous frame, for each conversion coefficient of each divided region of the encoding target frame An inter-frame prediction step of searching for a transform coefficient having a high correlation with each other, taking a difference between the transform coefficient of each divided region and the found transform coefficient, and outputting difference data, and encoding for encoding the difference data Process.

  According to the above configuration, when a moving image is encoded using motion prediction, it is possible to suppress image quality deterioration of an inter frame.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

(First embodiment)
As shown in FIG. 1, a moving image to be processed in the present invention is composed of an image and a sound, and the image is composed of a frame representing information of an instantaneous moment.

  FIG. 2 is a block diagram illustrating a configuration of the moving image processing apparatus according to the first embodiment. In the figure, 200 is a CPU, 201 is a memory, 202 is a terminal, 203 is a storage unit, 204 is an imaging unit, 205 is a display unit, and 206 is an encoding unit.

<Description of Processing of Encoding Unit 206>
Next, frame data encoding processing in the encoding unit 206 will be described with reference to the configuration of the encoding unit 206 shown in FIG. 3 and the flowchart of FIG. The details of the header creation method and the like are as described in the ISO / IEC recommendation, and will not be described here.

  In the following description, it is assumed that the frame data to be encoded is 8-bit monochrome frame data. However, the form of the frame data is not limited to this, and a monochrome image represented by a number of bits other than 8 bits such as 4 bits, 10 bits, and 12 bits for each pixel, or each color component (RGB / Lab) for each pixel. / YCrCb) can also be applied to color multivalued frame data expressing 8 bits. Further, the present invention can be applied to multi-value information representing the state of each pixel constituting an image, for example, multi-value index value representing the color of each pixel. When applied to these, each type of multi-value information may be monochrome frame data described later.

  First, pixel data constituting frame data of an image to be encoded is input from the imaging unit 204 to the frame data input unit 301 in the order of raster scan and output to the tile dividing unit 302.

  The tile dividing unit 302 divides one image input from the frame data input unit 301 into N tiles as shown in FIG. 5 (step S401), and identifies the first tile in order to identify each tile. In the embodiment, tile numbers 0, 1, 2,..., N−1 are assigned in the raster scan order. Hereinafter, data representing each tile is referred to as “tile data”. Although FIG. 5 shows an example in which an image is divided into 48 tiles of 8 horizontal by 6 vertical, it goes without saying that the number of divided tiles can be changed as appropriate. The generated tile data is sequentially sent to the discrete wavelet transform unit 303. In the processing after the discrete wavelet transform unit 303, encoding is performed for each tile data.

  Further, the ROI tile determination unit 317 determines which tile (ROI tile) is encoded with high image quality (step S402). FIG. 6 is a diagram illustrating an example of the determined ROI tile. Note that the ROI tile determination unit 317 determines an ROI tile for an area including a priority area specified by an input device (not shown) by the user. In step S403, a counter for recognizing the tile being processed is set to i = 0.

  Next, the frame attribute determining unit 316 determines whether the frame to be encoded is an I frame (Intra frame) or a P frame (Predictive frame) (step S404). If the encoding target frame is an I frame, the tile data is output to the discrete wavelet transform unit 303. On the other hand, if the encoding target frame is a P frame, the frame data is copied to a motion compensation (MC) prediction unit 310.

[When encoding target frame is I frame]
When the encoding target frame is an I frame, in step S405, the discrete wavelet transform unit 303 inputs a plurality of pixels (one tile data x (n) in the frame data of one frame image input from the tile dividing unit 302 ( Discrete wavelet transform is performed using reference pixel data (hereinafter referred to as “reference pixel data”).

Here, the frame data (discrete wavelet transform coefficient) after the discrete wavelet transform is shown.
Y (2n) = X (2n) + floor {(Y (2n-1) + Y (2n + 1) +2) / 4}
Y (2n + 1) = X (2n + 1) -floor {(X (2n) + X (2n + 2)) / 2} (1)

  Y (2n) and Y (2n + 1) are discrete wavelet transform coefficient sequences, Y (2n) is a low-frequency subband, and Y (2n + 1) is a high-frequency subband. In the conversion formula (1), floor {X} represents a maximum integer value not exceeding X. FIG. 7 schematically shows the discrete wavelet transform.

  The conversion equation (1) is for one-dimensional data. By applying this conversion in the order of the horizontal direction and the vertical direction, and performing two-dimensional conversion, the LL, as shown in FIG. It can be divided into four subbands HL, LH, and HH. Here, L indicates a low-frequency subband, and H indicates a high-frequency subband. Next, the LL subband is divided into four subbands in the same manner (FIG. 8B), and the LL subband is further divided into four subbands (FIG. 8C). In this way, a total of 10 subbands are created. Each of the ten subbands is called HH1, HL1,... As shown in FIG. Here, the number in the name of each subband indicates the level of each subband. That is, the level 1 subbands are HL1, HH1, and LH1, the level 2 subbands are HL2, HH2, and LH2, and the level 3 subbands are HL3, HH3, and LH3. The LL subband is a level 0 subband. Since there is only one LL subband, no subscript is added. A decoded image obtained by decoding subbands from level 0 to level n is referred to as a level n decoded image. The higher the level of the decoded image, the higher the resolution.

  The transform coefficients of 10 subbands are temporarily stored in the buffer 304, and are in the order of LL, HL1, LH1, HH1, HL2, LH2, HH2, HL3, LH3, and HH3, that is, from the subband having the lowest level to the higher level. The result is output to coefficient quantization section 305 in the order of subbands.

The coefficient quantization unit 305 quantizes the transform coefficient of each subband output from the buffer 304 at a quantization step determined for each frequency component, and the quantized value (coefficient quantization value) is an entropy coding unit. The data is output to 306 (step S406). When the coefficient value is X and the quantization step value for the frequency component to which the coefficient belongs is q, the quantized coefficient value Q (X) is obtained by the following equation (2).
Q (X) = floor {(X / q) +0.5} (2)

  FIG. 9 shows the correspondence between each frequency component and the quantization step in the present embodiment. As shown in the figure, a larger quantization step is given to a sub-band having a higher level. Note that the quantization step for each subband is stored in advance in a memory such as a RAM or a ROM (not shown). After all the transform coefficients in one subband are quantized, the coefficient quantized values are output to the entropy coding unit 306 and the inverse coefficient quantization unit 312.

The inverse coefficient quantization unit 312 uses the quantization step of FIG. 9 to inversely quantize the coefficient quantization value based on the following equation (3) (step S407).
Y = q * Q (3)

Here, q is a quantization step, Q is a coefficient quantization value, and Y is an inverse quantization value.
The inverse discrete wavelet transform unit 313 performs inverse discrete wavelet transform on the inverse quantized value according to the following equation (4) (step S408).
X (2n) = Y (2n) -floor {(Y (2n-1) + Y (2n + 1) +2) / 4}
X (2n + 1) = Y (2n + 1) + floor {(X (2n) + X (2n + 2)) / 2} (4)

  Then, the obtained decoded pixel is recorded in the frame memory 311 (step S409).

  On the other hand, the entropy encoding unit 306 performs entropy encoding on the input coefficient quantization value (step S410). Here, as shown in FIG. 10, first, each subband, which is a collection of input coefficient quantization values, is divided into rectangles (referred to as “code blocks”). The size of this code block is set to 2m × 2n (m and n are integers of 2 or more). Further, this code block is divided into bit planes as shown in FIG. Then, each bit in each bit plane is divided into three types based on a predetermined classification rule, as shown in FIG. 12, and three types of coding paths that collect the same type of bits are generated. These three types of coding passes are a significance propagation pass that is a sign path of insignificant coefficients around which a significant coefficient is present, a magnitude refinement pass that is a sign path of significant coefficients, and a sign path of the remaining coefficient information. cleanup pass.

  The input coefficient quantization value is subjected to binary arithmetic coding, which is entropy coding, with the coding pass obtained here as a unit, and an entropy coded value is generated.

  Note that entropy encoding is performed in order from the upper bit plane to the lower bit plane when attention is paid to one code block. Further, when attention is paid to a bit plane having the code block, the three types of paths shown in FIG. The encoding is performed in order. FIG. 12 shows the classification of coding paths in the fourth bit plane of FIG.

  The coding path subjected to entropy encoding is output to the tile encoded data generation unit 307.

  The tile encoded data generation unit 307 forms a single or a plurality of layers from a plurality of input coding passes, and generates tile encoded data using these layers as data units (step S411). The layer structure will be described below.

  As shown in FIG. 13, the tile encoded data generation unit 307 collects entropy-encoded coding paths from a plurality of code blocks in a plurality of subbands, and configures a layer. FIG. 13 shows a case where five layers are generated. When a coding pass is acquired from an arbitrary code block, as shown in FIG. 14, selection is always made in order from the coding pass that exists at the top in the code block. After that, as shown in FIG. 15, the tile encoded data generation unit 307 arranges the generated layers in order from the upper layer and generates a tile encoded data by adding a tile header to the head thereof. . This header stores information for identifying a tile, the code length of the encoded tile data, various parameters used for compression, and the like. The tile encoded data generated in this way is output to the frame encoded data generation unit 308.

  Next, in step S412, whether or not tile data to be encoded remains is determined by comparing the value of the counter i with the tile number. If tile data to be encoded remains (that is, i <N−1), the counter i is incremented by 1 in step S413, and the process returns to step S405 to repeat the processing up to step S412 for the next tile. If no tile data to be encoded remains (that is, i = N−1), the process proceeds to step S426.

  In step S426, the frame encoded data generation unit 308 arranges tile encoded data as shown in FIG. 15 in a predetermined order (for example, in order of tile numbers) as shown in FIG. Is added to generate frame encoded data. The header stores the input image, the vertical and horizontal sizes of tiles, various parameters used for compression, and the like. The frame encoded data generated in this way is output from the frame encoded data output unit 309 to the recording unit 212.

  In the above description, the processes of steps S407 to S409 are described as being performed prior to the processes of steps S410 and S411. However, the processes may be performed in the reverse order or in parallel.

[When encoding target frame is P frame]
Next, a process when the encoding target frame is a P frame in the determination in step S404 will be described. In that case, as described above, the tile division unit 302 copies the frame data to the MC prediction unit 310, and the MC prediction unit 310 determines whether the frame (previous frame) recorded in the frame memory 311 and the encoding target frame are included. MC prediction is performed between them (step S414). Here, as shown in FIG. 17, MC prediction destination data is limited to the ROI tile of the previous frame. This is to avoid image quality reduction of non-ROI tiles due to accumulation of data discard in the tile encoded data generation unit.

  The subtractor 314 calculates the difference between the previous frame and the encoding target frame based on the prediction result (step S415). As for the subtraction result (difference data) obtained there, discrete wavelet transform (step S416), quantization (step S417), inverse quantization (step S418), and inverse discrete wavelet transform (step S419), as in the case of processing for the I frame. Entropy encoding (step S422), tile encoded data generation (step S423), tile number determination (step S424), and image encoded data generation (step S426) are performed.

  The difference from the processing of the I frame is that the sum calculator 315 calculates the sum of the difference data and the previous frame, restores the encoding target frame (step S420), and stores the obtained decoded frame in the frame memory 311. This is a point that there is a recording process (step S421). In step S414 described above, MC prediction is performed using the decoded frame recorded here.

  The processes of steps S414 to S423 are repeatedly performed through the process of incrementing the counter i by 1 in step S425 until it is determined in step S424 that there is no tile data to be encoded.

  As a unit of data used in prediction, a tile or a block obtained by further dividing the tile can be considered.

  In FIG. 4, the processing of steps S418 to S421 is described as being performed prior to the processing of steps S422 and S423. However, the processing may be performed in the reverse order or in parallel.

  As described above, according to the first embodiment, by setting only the ROI tile in the previous frame as the MC prediction destination, it is possible to avoid image quality reduction of the P frame due to accumulation of data discard in the tile encoded data generation unit. Can do.

(Second Embodiment)
In the first embodiment, the method of avoiding the image quality reduction of the P frame due to accumulation of data discard in the tile encoded data generation unit by limiting to the ROI tile as the prediction destination data has been shown.

  In general, the user sets an object as an ROI, and a tile that includes the object is determined as an ROI tile. For this reason, the pixel distribution and characteristics of the ROI tile are similar between the following frames. For this reason, it is thought that the prediction between ROI tiles can implement | achieve high encoding efficiency. However, it is also conceivable that prediction between ROI tiles and non-ROI tiles cannot achieve such high coding efficiency. If the encoding efficiency is not so high, the MC prediction process can be wasted. Therefore, in the second embodiment, MC prediction is performed only between ROI tiles. Note that the second embodiment is different from the first embodiment only in the process in step S415 of the encoding process shown in FIG. 4, and only this point will be described.

  FIG. 18 shows the processing in the MC prediction unit 310 performed in step S415 in the second embodiment. Here, as shown in FIG. 18, MC prediction is performed only between ROI tiles, and MC prediction of non-ROI tiles is not performed.

  As described above, in the second embodiment, by performing MC prediction only between ROI tiles, it is possible to avoid a reduction in image quality of P frames while omitting useless calculations.

(Third embodiment)
In the third embodiment, the ROI region is set on the discrete wavelet transform coefficient space without setting the ROI region for each tile. Then, by limiting to the ROI coefficient as a prediction destination, image quality reduction of the P frame is avoided.

  FIG. 19 is a block diagram of the encoding unit 206 in the third embodiment. Note that the moving image processing apparatus is the same as that shown in FIG. The configuration shown in FIG. 19 is obtained by replacing the ROI tile determination unit 317 with the ROI determination unit 417 as compared with the block diagram of the encoding unit 206 in the first embodiment. The ROI tile determining unit 317 determines a region in units of tiles, whereas the ROI determining unit 417 is different in that the region is determined in units of pixels. For example, the former ROI tile determination unit 317 determines a tile that includes a region extracted by an object extraction unit (not shown) as a ROI tile, while the latter ROI tile determination unit 417 determines the extracted region as an ROI tile. The ROI area is determined in pixel units.

  Further, the position of the subtraction unit 314 is changed due to the change of the data to be predicted from the pixel to the coefficient, the point that the ROI unit 418 and the inverse ROI unit 419 are added, and the inverse discrete wavelet transform unit 313 is no longer necessary. There are differences. The inverse ROI unit 419 processes the coefficients from (c) to (a) in FIG.

  FIG. 20 is a flowchart showing the encoding process in the third embodiment. The same processes as those in the flowchart of FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted.

[When encoding target frame is I frame]
In the third embodiment, when the encoding target frame is an I frame, after quantizing the transform coefficient transformed by the discrete wavelet transform unit 303 (step S406), whether or not the ROI unit 418 is an ROI in step S506. Accordingly, the coefficient quantization value is changed based on the following equation (5).
Q "= Q * 2 ^B ; (Q: absolute value of coefficient quantization value obtained from pixels in ROI)
Q '= Q; (Q: absolute value of coefficient quantization value other than above) (5)

Here, B is given for each subband, and each Q ′ is set to be larger than any Q ″ in the target subband. That is, the original coefficient quantization value of Q ′ is configured. Therefore, the bits are shifted up so that the bits constituting the original coefficient quantized value of Q "and the bits constituting Q" do not exist in the same digit.
With the above processing, only the coefficient quantization value related to the ROI is shifted up by B bits.
FIG. 21A shows ROI and non-ROI in each subband, and FIGS. 21B and 21C are conceptual diagrams showing changes in coefficient quantization values due to shift-up. In FIG. 21B, three coefficient quantized values exist in each of the three subbands, and the coefficient quantized values that are shaded are the coefficient quantized values constituting the ROI. After shifting up, they become as shown in FIG.

  In step S507, the reverse ROI unit 419 performs a process of shifting down the ROI region whose bits are shifted up by the ROI unit 418.

[When encoding target frame is P frame]
When the encoding target frame is a P frame, in the third embodiment, first, discrete wavelet transform is performed in step S514. Thereafter, MC prediction is performed on the discrete wavelet transform coefficient space in step S515. Here, the MC prediction unit 310 limits only the DWT coefficient related to the ROI coefficient to the prediction target data as shown in FIG.

  In step S516, the difference (difference data) between the previous frame and the encoding target frame is calculated based on the prediction result. The coefficient quantizing unit 305 quantizes the difference data (step S417). Thereafter, in step S517, the ROI unit 418 changes the coefficient quantization value of the difference data based on the above-described equation (5) depending on whether or not the ROI.

  In step S518, the reverse ROI unit 419 performs a process of shifting down the ROI region whose bits are shifted up by the ROI unit 418.

  As described above, in the third embodiment, it is possible to avoid a reduction in the image quality of the P frame by performing MC prediction using only the coefficients related to the ROI.

(Other embodiments)
In the first to third embodiments, the invention has been described with respect to the discrete wavelet transform. However, embodiments to which the discrete cosine transform is applied are also within the scope of the present invention.

  Further, even if the present invention is applied as part of a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), a single device (for example, a copying machine) The present invention may be applied to a part of an apparatus including a digital camera.

  Another object of the present invention is to supply a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and the computer (or CPU or Needless to say, this can also be achieved by the MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included. Examples of the storage medium for storing the program code include a flexible disk, hard disk, ROM, RAM, magnetic tape, nonvolatile memory card, CD-ROM, CD-R, DVD, optical disk, magneto-optical disk, MO, and the like. Can be considered. Also, a computer network such as a LAN (Local Area Network) or a WAN (Wide Area Network) can be used to supply the program code.

  Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts described above.

It is a figure which shows the concept of the moving image of the encoding target in embodiment of this invention. It is a block diagram which shows the structure of the moving image processing apparatus in embodiment of this invention. It is a block diagram which shows the structure of the encoding part in the 1st Embodiment of this invention. It is a flowchart which shows the encoding process in the 1st Embodiment of this invention. It is explanatory drawing of tile division | segmentation. It is a figure which shows an example of a ROI tile. It is explanatory drawing of a one-dimensional discrete wavelet transform. (A) is a diagram that decomposes into four subbands, (b) is a diagram that further decomposes the LL subband of (a) into four subbands, and (c) is a diagram that further decomposes the LL subband of (b) into four subbands. It is a figure which decomposes | disassembles into a subband. It is explanatory drawing of a quantization step. It is explanatory drawing of a code block division | segmentation. It is explanatory drawing of bit-plane division | segmentation. It is explanatory drawing of a coding pass. It is explanatory drawing of a layer production | generation. It is explanatory drawing of a layer production | generation. It is explanatory drawing of a structure of tile coding data. It is explanatory drawing of a structure of frame coding data. It is a figure which shows the concept of the data of MC prediction destination in the 1st Embodiment of this invention. It is a figure which shows the concept of the data of MC prediction destination in the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the encoding part in the 3rd Embodiment of this invention. It is a flowchart which shows the encoding process in the 3rd Embodiment of this invention. (A) shows ROI and non-ROI in each subband, and (b) and (c) are conceptual diagrams showing changes in coefficient quantization values due to shift-up. It is a figure which shows the concept of the data of MC prediction destination in the 3rd Embodiment of this invention. It is a block diagram which shows the encoding part by a JPEG2000 encoding system. It is a block diagram which shows the decoding part by a JPEG2000 encoding system. It is a block diagram which shows the encoding part by an MPEG encoding system. It is a figure which shows the concept of the data of the conventional MC prediction destination.

Explanation of symbols

DESCRIPTION OF SYMBOLS 301 Frame data input part 302 Tile division part 303 Discrete wavelet transformation part 304 Buffer 305 Coefficient quantization part 306 Entropy encoding part 307 Tile encoded data generation part 308 Frame encoded data generation part 309 Frame encoded data output part 310 Motion compensation (MC) Prediction unit 311 Frame memory 312 Inverse coefficient quantization unit 313 Inverse discrete wavelet transform unit 314 Subtractor 315 Sum calculator 316 Frame attribute determination unit 317 ROI tile determination unit 417 ROI determination unit 418 ROI unit

Claims (17)

A video encoding device that encodes a video using inter-frame motion prediction,
A dividing means for dividing each frame into a plurality of divided regions;
A determination means for determining an important area from within the frame;
Within the important area of the previous frame, a pixel set having a high correlation is searched for each divided area of the encoding target frame, and the difference between the data of each divided area and the data of the searched pixel set is obtained. Inter-frame prediction means for outputting difference data by
A moving picture coding apparatus comprising: coding means for coding the difference data.   2. The moving picture encoding apparatus according to claim 1, wherein the encoding unit discards data with priority from an area outside the important area in order to adjust a code amount. A determination unit for determining whether a frame to be encoded is a frame to be intra-frame encoded or a frame to be encoded between frames;
When the determination unit determines that the frame is an intra-frame encoded frame, the encoding unit encodes the data of each divided region of the encoding target frame without performing the processing by the inter-frame prediction unit. The moving picture coding apparatus according to claim 1 or 2, wherein   The moving image according to any one of claims 1 to 3, wherein the inter-frame prediction unit performs processing only on an important region determined by the determination unit among the divided regions of the encoding target frame. Image encoding device.   The moving image encoding apparatus according to claim 1, wherein the encoding unit performs discrete wavelet transform.   6. The moving image encoding apparatus according to claim 5, wherein the encoding unit performs encoding according to a JPEG2000 encoding method.   The moving image encoding apparatus according to claim 1, wherein the encoding unit performs discrete cosine transform. A video encoding device that encodes a video using inter-frame motion prediction,
A dividing means for dividing each frame into a plurality of divided regions;
A determination means for determining an important area from within the frame;
A conversion means for performing data conversion for each divided region and generating a conversion coefficient;
From the transform coefficients corresponding to the range of the important area of the previous frame, a transform coefficient having a high correlation is searched for each transform coefficient of each divided area of the frame to be encoded, and the transform coefficient of each divided area and the searched transform An inter-frame prediction means for taking a difference from a coefficient and outputting difference data;
A moving picture coding apparatus comprising: coding means for coding the difference data.   9. The moving picture coding apparatus according to claim 8, wherein the coding unit discards data in preference to an area outside the important area in order to adjust a code amount. A determination unit for determining whether a frame to be encoded is a frame to be intra-frame encoded or a frame to be encoded between frames;
When the determination unit determines that the frame is an intra-frame encoded frame, the encoding unit does not perform the process by the inter-frame prediction unit, and the encoding unit calculates a transform coefficient for each divided region of the encoding target frame. The moving image encoding apparatus according to claim 8 or 9, wherein encoding is performed.   The inter-frame prediction unit performs processing only on transform coefficients of an important region determined by the determination unit among divided regions of the encoding target frame. Video encoding device.   The moving image encoding apparatus according to claim 8, wherein the conversion unit performs discrete wavelet conversion.   The moving image encoding apparatus according to claim 8, wherein the conversion unit performs discrete cosine conversion. A moving image encoding method for encoding a moving image using inter-frame motion prediction,
A dividing step of dividing each frame into a plurality of divided regions;
A decision process to determine important areas from within the frame;
Within the important area of the previous frame, a pixel set having a high correlation is searched for each divided area of the encoding target frame, and a difference between the data of each divided area and the searched pixel set data is obtained. Inter-frame prediction process for outputting difference data
And a coding process for coding the difference data. A moving image encoding method for encoding a moving image for each frame using inter-frame motion prediction,
A dividing step of dividing each frame into a plurality of divided regions;
A decision process to determine important areas from within the frame;
A conversion step of performing data conversion for each divided area and generating a conversion coefficient;
From the transform coefficients corresponding to the range of the important area of the previous frame, a transform coefficient having a high correlation is searched for each transform coefficient of each divided area of the frame to be encoded, and the transform coefficient of each divided area and the searched transform An inter-frame prediction step of taking the difference with the coefficient and outputting the difference data;
And a coding process for coding the difference data.   A program executable by an information processing apparatus, comprising program code for realizing the moving picture coding method according to claim 14.   A storage medium readable by an information processing apparatus, wherein the program according to claim 16 is stored.

Images