JP4906458B2 - Image coding apparatus and image coding method - Google Patents

Description

  The present invention relates to an encoding technique using interframe motion compensation.

  Conventionally, compression encoding methods for moving images are MPEG-1, 2, 4 and H.264. There are standards such as H.264. The compression encoding process for the moving image is performed by dividing an original image (image) included in the moving image into predetermined regions called blocks, and performing motion compensation prediction and DCT conversion processing in units of the divided blocks. It is.

  In the case of motion compensation prediction, the block size as a unit of compression encoding processing is 16 horizontal pixels × 16 vertical lines called macroblocks in the MPEG1 and MPEG2 systems. When frame prediction is performed for this one macroblock, one motion vector composed of two components in the horizontal direction and the vertical direction is assigned. When performing field prediction, two motion vectors composed of two components in the horizontal direction and the vertical direction are assigned.

  As shown in FIG. 2 (a), the motion vector assignment processing when performing motion compensation prediction using macroblocks as in units of MPEG1 and MPEG2 uses macroblocks 201 as processing units. Assume that all the pixels in the macroblock 201 are represented by the same motion vector 202. Then, the moving image is processed by the movement amount corresponding to each of the horizontal direction and the vertical direction of the motion vector 202.

  In the MPEG4 system, in addition to performing motion compensation prediction in units of macroblocks, motion compensation is performed in units of blocks (small blocks) 203 having a size of horizontal 8 pixels × vertical 8 lines, as shown in FIG. 2B. Mode (8 × 8 mode) is provided. When this 8 × 8 mode is used, when a plurality of motions exist in the macro block 204 of 16 horizontal pixels × 16 vertical lines, rather than assigning one motion vector to one of the macro blocks 204, It is possible to obtain a motion vector that approximates the actual motion.

  For example, as shown in FIG. 3A, the background (the tower and the sun) is stationary, and the movement of the car and the background portion is different when considering an image in which the car is running in the left direction. In such an image, as shown in FIG. 3B, when a part of the vehicle and the background are mixed in the same macroblock 301, motion prediction 302 is performed by giving a motion vector 302 for the vehicle and a motion vector 303 for the background. Efficiency is improved. However, when dividing into small blocks, there are four small blocks for one macroblock, so the number of motion vectors is one for each small block. There will be one.

  H. In the H.264 system, as shown in FIG. 4A, the macro block can be a rectangular block other than a square block of horizontal 16 pixels × vertical 16 lines and horizontal 8 pixels × vertical 8 lines. For example, a rectangular block such as horizontal 16 pixels × vertical 8 lines and horizontal 8 pixels × vertical 16 lines can be formed, or a macro block can be divided into four blocks as small blocks of horizontal 8 pixels × vertical 8 lines. It is.

  Furthermore, this block is divided into the size of horizontal 8 pixels × vertical 4 lines, horizontal 4 pixels × vertical 8 lines, horizontal 4 pixels × vertical 4 lines only for this block of horizontal 8 pixels × vertical 8 lines. It is also possible to perform motion compensation prediction in blocks. In this method, there is an INTRA prediction mode, and it is possible to predict the pixel value of the macroblock from the image information in the screen. In this mode, as shown in FIG. 4B, the macroblock can be predicted with a block divided into a size of horizontal 16 pixels × vertical 16 lines and horizontal 4 pixels × vertical 4 lines. .

By dividing the macroblock into smaller sub-macroblocks and performing motion compensation prediction, it is possible to express a fine motion vector that matches the actual motion. However, on the other hand, since additional information such as vector information is required for each divided sub-macroblock, it is not always efficient for encoding to divide into sub-macroblocks. Therefore, it is necessary to perform encoding by selecting a combination of sub-macroblocks of an optimal size from among blocks of various sizes (see Patent Document 1).
JP 2005-012439 A

  In the prior art, in order to select the optimal combination of block sizes from sub-macroblocks of various sizes, the calculation error is calculated and compared for all combinations of various block sizes, so a huge amount of computation is required. Needed. Therefore, the encoding apparatus is required to increase hardware and process with a high-speed clock, and there is a problem that hinders downsizing of the apparatus and low power consumption.

  The present invention has been made in view of the above problems, and an object thereof is to provide a technique for reducing the amount of calculation in motion compensation predictive coding.

  In order to achieve the object of the present invention, for example, an image encoding apparatus of the present invention comprises the following arrangement.

That is, an image encoding device that encodes image data of each frame constituting a moving image,
Input means for inputting the image data of the encoding target frame for each macroblock;
Storage means for storing, as image data of a plurality of reference frames, image data of a plurality of frames input in the past from the encoding target frame;
Difference amount of pixel value between encoding target macroblock in encoding target frame and macroblock at position in reference frame corresponding to position of encoding target macroblock in encoding target frame A calculation means for obtaining
A determination unit that determines an encoding content for the encoding target macroblock by comparing the difference amount with a first threshold value set in advance and a second threshold value that is larger than the first threshold value. And
When the difference amount is less than the first threshold, the encoding target macroblock is determined as a block for which motion compensation is not performed,
If the difference amount is greater than or equal to the first threshold and less than the second threshold, the encoding target macroblock is determined as a block that performs motion compensation without being divided into sub-macroblocks;
If the difference amount is greater than or equal to the second threshold value, and said determining means for determining the block for motion compensation the encoded macroblock, after divided into sub-macroblock,
When the determining unit determines that the encoding target macroblock is a block that performs motion compensation without being divided into sub-macroblocks, a macroblock similar to the encoding target macroblock is determined as the plurality of reference frames. Each of the image data is subjected to motion compensation in units of macroblocks using motion vectors detected between the searched macroblock and the encoding target macroblock, and the encoding target macroblock is determined. First encoding means for encoding;
When the determination unit determines the block to be subjected to motion compensation after dividing the encoding target macroblock into sub-macroblocks, the encoding target macroblock is divided into a plurality of sub-macroblocks and divided. Submacroblocks similar to the respective submacroblocks are searched from the respective image data of the plurality of reference frames, and the submacroblocks searched for the respective divided submacroblocks and the respective divided submacroblocks Second encoding means for encoding each of the divided sub-macroblocks by performing motion compensation in units of sub-macroblocks using motion vectors detected between
When the determination unit determines the encoding target macroblock to be a block that does not perform motion compensation, the information indicating no motion vector is encoded without performing a motion search of the encoding target macroblock. And 3 encoding means.

  In order to achieve the object of the present invention, for example, an image encoding method of the present invention comprises the following arrangement.

That is, an image encoding method performed by an image encoding device that encodes image data of each frame constituting a moving image,
An input step of inputting image data of the encoding target frame for each macroblock;
Storing image data of a plurality of frames input in the past from the encoding target frame as image data of a plurality of reference frames;
Difference amount of pixel value between encoding target macroblock in encoding target frame and macroblock at position in reference frame corresponding to position of encoding target macroblock in encoding target frame A calculation process for obtaining
This is a determination step of determining the encoding content for the encoding target macroblock by comparing the difference amount with a first threshold value set in advance and a second threshold value larger than the first threshold value. And
When the difference amount is less than the first threshold, the encoding target macroblock is determined as a block for which motion compensation is not performed,
If the difference amount is greater than or equal to the first threshold and less than the second threshold, the encoding target macroblock is determined as a block that performs motion compensation without being divided into sub-macroblocks;
If the difference amount is greater than or equal to the second threshold value, the encoding target macro-blocks, and the determination step of determining a block to be motion-compensated from divided into sub-macroblock,
If the encoding target macroblock is determined to be a block that performs motion compensation without being divided into sub-macroblocks in the determining step, a macroblock similar to the encoding target macroblock is determined as the plurality of reference frames. Each of the image data is subjected to motion compensation in units of macroblocks using motion vectors detected between the searched macroblock and the encoding target macroblock, and the encoding target macroblock is determined. A first encoding step for encoding;
In the determination step, when the encoding target macroblock is determined to be a block to be subjected to motion compensation after being divided into sub macroblocks, the encoding target macroblock is divided into a plurality of submacroblocks and divided. Submacroblocks similar to the respective submacroblocks are searched from the respective image data of the plurality of reference frames, and the submacroblocks searched for the respective divided submacroblocks and the respective divided submacroblocks A second encoding step of encoding each of the divided sub-macroblocks by performing motion compensation in units of sub-macroblocks using motion vectors detected between
When the encoding target macroblock is determined as a block for which motion compensation is not performed in the determining step, information indicating no motion vector is encoded without performing a motion search for the encoding target macroblock. And 3 encoding steps.

  According to the configuration of the present invention, it is possible to reduce the amount of calculation in motion compensation predictive coding.

  Hereinafter, the present invention will be described in detail according to preferred embodiments with reference to the accompanying drawings.

[First Embodiment]
FIG. 1 is a block diagram illustrating a functional configuration of the image encoding device according to the present embodiment. Such an image encoding device is for encoding an image of each frame. More specifically, see H.C. As in H.264, one frame image is divided into a plurality of macroblocks, and motion prediction is performed for each divided macroblock and encoded. When one macro block is divided into a plurality of sub macro blocks (small blocks), motion prediction is performed in units of sub macro blocks.

  As shown in the figure, the image coding apparatus includes a frame buffer 101, a small block determination unit 102, a motion prediction unit 105, a motion compensation unit 106, an intra prediction unit 107, an orthogonal transform unit 108, a quantization unit 109, an entropy code. A quantization unit 110 and an inverse quantization unit 111. Furthermore, an inverse orthogonal transform unit 112, a switch unit 113, a reference frame buffer 114, and adders 103 and 104 are included. In the present embodiment, each unit illustrated in FIG. 1 will be described as being configured by hardware.

  Hereinafter, the operation of each unit will be described.

  The image of each frame constituting the moving image to be encoded is sequentially input and stored in the frame buffer 101.

The image of each frame, macro block (16 horizontal pixels × 16 vertical lines) in the small block determination section 102, a motion prediction unit 105, is inputted to the intra prediction unit 107.

The intra prediction unit 107 receives the respective macro blocks constituting the target frame from the frame buffer 1 01, for each macroblock, the prediction image using the pixel values of the neighboring blocks is already encoded in the same space (Prediction data) is generated. For example, the intra prediction unit 107 performs the first frame among the frames constituting the moving image. The generated predicted image is sent to the adders 103 and 104 via the switch unit 113 at the subsequent stage.

The adder 103 generates a difference amount between the macroblock input from the frame buffer 101 and the prediction data for the macroblock as prediction error data, and sends the prediction error data to the subsequent orthogonal transform unit 108. The operation of the adder 104 will be described later.

  Also, the intra prediction unit 107 outputs encoding information such as a pixel pattern used for intra prediction encoding to the entropy encoding unit 110.

  On the other hand, the small block determination unit 102 determines an encoding unit for the macroblock (macroblock P) in the encoding target frame input from the frame buffer 101. First, the small block determination unit 102 first selects a macroblock (positionally corresponding to the macroblock P) in the image of the frame before the encoding target frame from the reference frame buffer 114 (a frame before one or more arbitrary frames). Read macroblock Q). Then, the difference amount D between the macroblock P and the macroblock Q is calculated.

  Here, if the pixel i constituting the macroblock P is Pi (1 ≦ i ≦ M × N) and the pixel j constituting the macroblock Q is Qj (1 ≦ j ≦ M × N), the difference amount D is as follows: It can obtain | require according to the formula of.

  As can be seen from the equation, it can be said that the smaller the difference amount D, the smaller the change between the macroblock P and the macroblock Q, and the larger the difference amount D, the greater the change.

  Next, the small block determination unit 102 performs a size comparison between the difference amount D and a preset threshold value th1, and determines that “no motion compensation is performed” if D <th1.

  Further, the small block determination unit 102 compares the difference amount D with the threshold th1 and the threshold th2 that is larger than the threshold th1. As a result, if th1 ≦ D <th2, the coding unit for the macroblock P is determined as “macroblock P”. That is, it is determined that the macroblock P is not divided into sub-macroblocks and is encoded by motion compensation in units of macroblocks.

  The small block determination unit 102 compares the difference amount D with the threshold th2. As a result, when D ≧ th2, the encoding unit for the macroblock P is determined as “a unit smaller than the macroblock P”. That is, it is determined that the macroblock P is divided into sub-macroblocks and is encoded by motion compensation in units of sub-macroblocks.

  By the processing by the small block determination unit 102, the encoding unit for the macroblock to be encoded can be determined according to the difference amount between the corresponding macroblocks.

  The determination result by the small block determination unit 102 is sent to the motion prediction unit 105 as determination information. The motion prediction unit 105 receives the determination information from the small block determination unit 102 and acquires the macroblock P and the past plural frames of image data stored in the reference frame buffer 114 from the frame buffer 101.

  Then, the motion prediction unit 105 switches processing as described below according to the determination information received from the small block determination unit 102.

  When the determination information “no motion compensation is performed” is received, the motion prediction unit 105 does not perform a motion search, and outputs encoded information indicating no motion vector to the motion compensation unit 106 and the entropy encoding unit 110.

  When receiving the determination information indicating that “macroblock P” is determined as the encoding unit for the macroblock P, the motion prediction unit 105 operates as follows. First, the motion prediction unit 105 searches for a macro block R similar to the macro block P from a plurality of past frames stored in the reference frame buffer 114. Then, a motion vector that is a spatial displacement amount is detected between the macroblock P and the macroblock R. Then, the motion prediction unit 105 performs motion compensation on the encoded information including the detected motion vector, the encoding unit (in this case, the macroblock P), and the number of the frame (reference frame) from which the macroblock R is extracted. Output to the unit 106 and the entropy encoding unit 110.

  When receiving the determination information indicating that “sub-macroblock” is determined as the encoding unit for the macroblock P, the motion prediction unit 105 operates as follows. First, the motion prediction unit 105 divides the macro block P into a plurality of sub macro blocks. Then, the following processing is performed for each sub macroblock (submacroblock S).

  First, the motion prediction unit 105 searches for a sub-macroblock T similar to the sub-macroblock S from a plurality of past frames stored in the reference frame buffer 114. Then, a motion vector that is a spatial displacement amount is detected between the sub-macroblock S and the sub-macroblock T.

  Then, the process for obtaining the motion vector for each sub macroblock is performed for each sub macroblock constituting the macroblock P. Then, the motion prediction unit 105 encodes a motion vector detected for each sub macroblock, a coding unit (in this case, a sub macroblock), and a frame (reference frame) number from which the sub macroblock T is extracted. Is output to the motion compensation unit 106 and the entropy encoding unit 110. Note that the above processing performed for each sub macroblock is an example, and the present invention is not limited to this.

  Moreover, since the motion compensation performed for each macroblock and the motion compensation performed for each sub-macroblock described above are well-known techniques, further explanation is omitted.

  Next, when receiving the encoding information from the motion prediction unit 105, the motion compensation unit 106 acquires an image of a frame corresponding to the reference frame number included in the encoding information from the reference frame buffer 114. Then, a predicted image for each macro block (sub macro block) is generated with reference to the acquired image of the frame. The predicted image is sent to the switch unit 113. The switch unit 113 sends the predicted image to the adder 103.

The adder 103 obtains the difference amount between each macro block (sub macro block) input from the frame buffer 101 and the corresponding prediction image as prediction error data, and sends it to the orthogonal transform unit 108 at the subsequent stage.

  When the prediction error data is input, the orthogonal transform unit 108 performs a known orthogonal transform on the prediction error data to generate an orthogonal transform coefficient. Then, the generated orthogonal transform coefficient is sent to the quantization unit 109 in the subsequent stage.

  The quantization unit 109 generates a quantized orthogonal transform coefficient by performing a known quantization process on the input orthogonal transform coefficient, and sends the quantized orthogonal transform coefficient to the entropy encoding unit 110 and the inverse quantization unit 111 in the subsequent stage.

  The entropy encoding unit 110 performs well-known entropy encoding based on the quantized orthogonal transform coefficient input from the quantization unit 109 and the encoded information input from the motion prediction unit 105, and multiplexes the compressed stream. , Send it out. A series of encoding processes performed by the entropy encoding unit 110 is described in H.264. Since this is in accordance with H.264, a description thereof will be omitted.

  On the other hand, the inverse quantization unit 111 generates an orthogonal transform coefficient by performing inverse quantization on the quantized orthogonal transform coefficient input from the quantization unit 109, and sends the orthogonal transform coefficient to the inverse orthogonal transform unit 112.

  The inverse orthogonal transform unit 112 generates prediction error data by performing inverse orthogonal transform on the orthogonal transform coefficient input from the inverse quantization unit 111, and sends the prediction error data to the adder 104.

  The adder 104 adds the prediction error data input from the inverse orthogonal transform unit 112 and the prediction image input from the switch unit 113 to generate a frame image. The frame image is filtered by a loop filter (not shown) and then sent to the reference frame buffer 114. This frame image is stored in the reference frame buffer 114 and used as a reference frame in the subsequent encoding process. In addition, information regarding motion vectors and reference frame numbers is also stored in the reference frame buffer 114 together with the reference frame image as additional data for the reference frame image.

  The series of encoding processes described above will be described using the flowchart shown in FIG. FIG. 5 is a flowchart of the encoding process for the macroblock P in the encoding target frame. Therefore, if the process according to the flowchart of FIG. 5 is performed for each macroblock constituting the encoding target frame, the encoding target frame can be encoded. Of course, if the encoding process for the encoding target frame is performed for each frame constituting the moving image, the entire moving image can be encoded.

  First, in step S <b> 501, the small block determination unit 102 acquires from the frame buffer 101 a macroblock P as an encoding target macroblock in the encoding target frame.

  Next, in step S502, the small block determination unit 102 first reads the macroblock Q from the reference frame buffer 114. Then, the small block determination unit 102 calculates the difference amount D between the macroblock P and the macroblock Q according to the above equation 1.

  In step S503, the small block determination unit 102 compares the difference amount D with a preset threshold th1. As a result of such size comparison, if D <th1, the process proceeds to step S504, and if D ≧ th1, the process proceeds to step S506.

  In step S <b> 504, the small block determination unit 102 determines that “motion compensation is not performed” and sends determination information indicating the determination result to the motion prediction unit 105.

  In step S505, since the motion prediction unit 105 has received the determination information “no motion compensation is performed”, no motion search is performed, and the motion compensation unit 106 and the entropy encoding unit 110 indicate encoded information indicating no motion vector. And output.

  On the other hand, in step S506, the small block determination unit 102 performs a size comparison between the difference amount D, the threshold th1, and the threshold th2 that is larger than the threshold th1. As a result of such size comparison, if th1 ≦ D <th2, the process proceeds to step S507, and if D> th2, the process proceeds to step S510.

  In step S <b> 507, the small block determination unit 102 determines the coding unit for the macroblock P as “macroblock P”, and sends determination information indicating the determination result to the motion prediction unit 105.

  In step S <b> 508, the motion prediction unit 105 first searches the macro block R similar to the macro block P from the past plural frames of images stored in the reference frame buffer 114. Then, a motion vector that is a spatial displacement amount is detected between the macroblock P and the macroblock R.

  In step S509, the motion prediction unit 105 converts the encoded information including the detected motion vector, the encoding unit (macroblock P), and the frame (reference frame) number from which the macroblock R is extracted into the motion compensation unit. 106 and the entropy encoding unit 110.

  On the other hand, since D ≧ th2 in step S510, the small block determination unit 102 determines that the encoding unit for the macroblock P is “a unit smaller than the macroblock P”. Then, determination information indicating the determination result is sent to the motion prediction unit 105.

  In step S511, the motion prediction unit 105 first divides the macro block P into a plurality of sub macro blocks. Then, the following processing is performed for each sub macroblock (submacroblock S).

  First, the motion prediction unit 105 searches for a sub-macroblock T similar to the sub-macroblock S from a plurality of past frames stored in the reference frame buffer 114. Then, a motion vector that is a spatial displacement amount is detected between the sub-macroblock S and the sub-macroblock T. Then, the process for obtaining the motion vector for each sub macroblock is performed for each sub macroblock constituting the macroblock P.

  In step S512, the motion prediction unit 105 performs encoding including a motion vector detected for each sub macroblock, an encoding unit (sub macroblock), and a frame (reference frame) number from which the sub macroblock T is extracted. Generate information. Then, the generated encoded information is output to the motion compensation unit 106 and the entropy encoding unit 110.

  Then, after any of steps S505, S509, and S512, the process proceeds to step S513.

  In step S513, the motion compensation unit 106 first acquires an image of a frame corresponding to the reference frame number included in the encoded information received from the motion prediction unit 105 from the reference frame buffer 114. Then, the motion compensation unit 106 generates a prediction image for each macro block (sub macro block) with reference to the acquired image of the frame. The predicted image is sent to the switch unit 113. The switch unit 113 sends the predicted image to the adder 103.

Next, in step S514 the adder 103, a macro block P (sub-macroblock S) inputted from the frame buffer 1 01 calculates a difference amount between corresponding predicted image as the prediction error data, the subsequent orthogonal transformation unit 108 To send.

  In step S515, the orthogonal transform unit 108 generates an orthogonal transform coefficient by performing known orthogonal transform on the prediction error data. Then, the generated orthogonal transform coefficient is sent to the quantization unit 109 in the subsequent stage.

  In step S516, the quantization unit 109 generates a quantized orthogonal transform coefficient by performing a known quantization process on the input orthogonal transform coefficient, and sends the quantized orthogonal transform coefficient to the entropy encoding unit 110 and the inverse quantization unit 111 in the subsequent stage. Send it out.

  In step S517, the entropy encoding unit 110 performs well-known entropy encoding based on the quantized orthogonal transform coefficient input from the quantization unit 109 and the encoded information input from the motion prediction unit 105, and generates a compressed stream. To multiplex. In step S518, this is sent to the outside.

  In step S519, the inverse quantization unit 111 performs inverse quantization on the quantized orthogonal transform coefficient input from the quantization unit 109 to generate an orthogonal transform coefficient, and sends the orthogonal transform coefficient to the inverse orthogonal transform unit 112.

  In step S <b> 520, the inverse orthogonal transform unit 112 generates prediction error data by performing inverse orthogonal transform on the orthogonal transform coefficient input from the inverse quantization unit 111, and sends the prediction error data to the adder 104.

  In step S521, the adder 104 generates a frame image by adding the prediction error data input from the inverse orthogonal transform unit 112 and the predicted image input from the switch unit 113.

  In step S522, the frame image is filtered by a loop filter (not shown) and stored in the reference frame buffer 114. In addition, information regarding motion vectors and reference frame numbers is also stored in the reference frame buffer 114 together with the reference frame image as additional data for the reference frame image.

  As described above, according to the present embodiment, it is possible to determine whether or not to perform motion compensation prediction before selecting an optimal combination of block sizes, and in that case, to determine the encoding unit. Depending on the image to be processed, the processing amount can be greatly reduced.

  For example, in the case of an image block with little motion, such as a background image taken with a fixed camera, the amount of difference between the encoding target frame and the previous frame is extremely small, so motion compensation with a small block is not possible. Not performed. Thereby, the processing amount can be reduced.

[Second Embodiment]
1 may be realized by software, and the rest may be realized by hardware. In this case, for example, the hardware is realized as a function expansion card that can be inserted into a personal computer, and the function expansion card is inserted into the personal computer. The software is stored on a memory included in the personal computer. According to such a configuration, the CPU included in the personal computer executes this software and also controls the operation of the function expansion card, whereby the processing described in the first embodiment (processing according to the flowchart of FIG. 5). ) Can be performed.

  FIG. 6 is a block diagram showing the hardware configuration of this computer.

  Reference numeral 601 denotes a CPU which controls the entire computer using programs and data stored in the RAM 602 and ROM 603.

  A RAM 602 has an area for temporarily storing programs and data loaded from the external storage device 606 and programs and data received from the outside via an I / F (interface) 607. Furthermore, it has a work area used when the CPU 601 executes various processes. That is, the RAM 602 can provide various areas as appropriate.

  Reference numeral 603 denotes a ROM which stores setting data and a boot program for the computer.

  An operation unit 604 includes a keyboard and a mouse, and various instructions can be input to the CPU 601 by an operator of the computer.

  A display unit 605 includes a CRT, a liquid crystal screen, and the like, and can display a processing result by the CPU 601 using an image, text, or the like.

  Reference numeral 606 denotes an external storage device, which is a large-capacity information storage device represented by a hard disk or the like. Here, an OS (operating system), programs and data for causing the CPU 601 to execute various processes performed by the computer, and the like. Is saved. The program and data include the above-described software and the operation control program for the function expansion card 608. The external storage device 606 also stores one or more moving image files to be encoded, and programs and data received from the outside via the I / F 607.

  Various types of information stored in the external storage device 606 are appropriately loaded into the RAM 602 under the control of the CPU 601. The CPU 601 executes processing using the loaded program and data, so that the computer can execute the encoding processing based on inter-frame motion compensation described in the first embodiment.

  Reference numeral 607 denotes an I / F for connecting the computer to a network such as a LAN or the Internet. For example, when a device that holds a moving image file is connected to the network, the computer can acquire a moving image file from the device via the I / F 607.

  Reference numeral 608 denotes a function expansion card, which is, for example, a processing board that performs part or all of the encoding processing by inter-frame motion compensation on the acquired moving image file.

  Reference numeral 609 denotes a bus connecting the above-described units.

[Other Embodiments]
Needless to say, the object of the present invention can be achieved as follows. That is, a recording medium (or storage medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

  Further, by executing the program code read by the computer, an operating system (OS) or the like running on the computer performs part or all of the actual processing based on the instruction of the program code. Needless to say, the process includes the case where the functions of the above-described embodiments are realized.

  Furthermore, it is assumed that the program code read from the recording medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. After that, based on the instruction of the program code, the CPU included in the function expansion card or function expansion unit performs part or all of the actual processing, and the function of the above-described embodiment is realized by the processing. Needless to say.

  When the present invention is applied to the recording medium, program code corresponding to the flowchart described above is stored in the recording medium.

It is a block diagram which shows the function structure of the image coding apparatus which concerns on the 1st Embodiment of this invention. It is a figure shown about allocation of a motion vector. It is a figure shown about the motion vector in a macroblock. It is a figure explaining a macroblock and a small block It is a flowchart of the encoding process with respect to the macroblock P in an encoding object frame. It is a block diagram which shows the hardware constitutions of a computer.

Claims (6)

An image encoding device that encodes image data of each frame constituting a moving image,
Input means for inputting the image data of the encoding target frame for each macroblock;
Storage means for storing, as image data of a plurality of reference frames, image data of a plurality of frames input in the past from the encoding target frame;
Difference amount of pixel value between encoding target macroblock in encoding target frame and macroblock at position in reference frame corresponding to position of encoding target macroblock in encoding target frame A calculation means for obtaining
A determination unit that determines an encoding content for the encoding target macroblock by comparing the difference amount with a first threshold value set in advance and a second threshold value that is larger than the first threshold value. And
When the difference amount is less than the first threshold, the encoding target macroblock is determined as a block for which motion compensation is not performed,
If the difference amount is greater than or equal to the first threshold and less than the second threshold, the encoding target macroblock is determined as a block that performs motion compensation without being divided into sub-macroblocks;
If the difference amount is greater than or equal to the second threshold value, and said determining means for determining the block for motion compensation the encoded macroblock, after divided into sub-macroblock,
When the determining unit determines that the encoding target macroblock is a block that performs motion compensation without being divided into sub-macroblocks, a macroblock similar to the encoding target macroblock is determined as the plurality of reference frames. Each of the image data is subjected to motion compensation in units of macroblocks using motion vectors detected between the searched macroblock and the encoding target macroblock, and the encoding target macroblock is determined. First encoding means for encoding;
When the determination unit determines the block to be subjected to motion compensation after dividing the encoding target macroblock into sub-macroblocks, the encoding target macroblock is divided into a plurality of sub-macroblocks and divided. Submacroblocks similar to the respective submacroblocks are searched from the respective image data of the plurality of reference frames, and the submacroblocks searched for the respective divided submacroblocks and the respective divided submacroblocks Second encoding means for encoding each of the divided sub-macroblocks by performing motion compensation in units of sub-macroblocks using motion vectors detected between
When the determination unit determines the encoding target macroblock to be a block that does not perform motion compensation, the information indicating no motion vector is encoded without performing a motion search of the encoding target macroblock. 3. An image encoding device comprising: 3 encoding means.   The calculation means includes an absolute value of a pixel value difference between the encoding target macroblock and a macroblock at a position in the reference frame corresponding to a position of the encoding target macroblock in the encoding target frame. The image encoding apparatus according to claim 1, wherein a result of summing up all the pixels is obtained as the difference amount.   The encoding by the encoding means is H.264. The image encoding apparatus according to claim 1, wherein the image encoding apparatus is an inter-frame motion compensation encoding according to H.264. An image encoding method performed by an image encoding device that encodes image data of each frame constituting a moving image,
An input step of inputting image data of the encoding target frame for each macroblock;
Storing image data of a plurality of frames input in the past from the encoding target frame as image data of a plurality of reference frames;
Difference amount of pixel value between encoding target macroblock in encoding target frame and macroblock at position in reference frame corresponding to position of encoding target macroblock in encoding target frame A calculation process for obtaining
This is a determination step of determining the encoding content for the encoding target macroblock by comparing the difference amount with a first threshold value set in advance and a second threshold value larger than the first threshold value. And
When the difference amount is less than the first threshold, the encoding target macroblock is determined as a block for which motion compensation is not performed,
If the difference amount is greater than or equal to the first threshold and less than the second threshold, the encoding target macroblock is determined as a block that performs motion compensation without being divided into sub-macroblocks;
If the difference amount is greater than or equal to the second threshold value, the encoding target macro-blocks, and the determination step of determining a block to be motion-compensated from divided into sub-macroblock,
If the encoding target macroblock is determined to be a block that performs motion compensation without being divided into sub-macroblocks in the determining step, a macroblock similar to the encoding target macroblock is determined as the plurality of reference frames. Each of the image data is subjected to motion compensation in units of macroblocks using motion vectors detected between the searched macroblock and the encoding target macroblock, and the encoding target macroblock is determined. A first encoding step for encoding;
In the determination step, when the encoding target macroblock is determined to be a block to be subjected to motion compensation after being divided into sub macroblocks, the encoding target macroblock is divided into a plurality of submacroblocks and divided. Submacroblocks similar to the respective submacroblocks are searched from the respective image data of the plurality of reference frames, and the submacroblocks searched for the respective divided submacroblocks and the respective divided submacroblocks A second encoding step of encoding each of the divided sub-macroblocks by performing motion compensation in units of sub-macroblocks using motion vectors detected between
When the encoding target macroblock is determined as a block for which motion compensation is not performed in the determining step, information indicating no motion vector is encoded without performing a motion search for the encoding target macroblock. And an encoding step. 3. An image encoding method comprising:   A computer program for causing a computer to function as the image encoding device according to any one of claims 1 to 3.   A computer-readable storage medium storing the program according to claim 5.

Images