JP2006157083A - Image coding apparatus, image coding method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an image with high quality by efficiently coding a moving image. <P>SOLUTION: A coefficient calculation unit 10 calculates a filter coefficient kq from a quantization scale QP obtained by a coding amount control unit 9, calculates a filter coefficient kb from a decoder buffer pointer DBP in a buffer 8, and selects a filter coefficient ke from the calculated coefficient kq or kb which is greater. A coefficient calculation unit 3 obtains a filter coefficient ka from a feature quantity measured by a feature quantity measurement unit 1. A filter control unit 4 obtains a filter control variable k on the basis of a result of comparing the filter coefficients ke with ka. A filter 5 places a band limit on a video signal by using the filter control variable. The filter 5 just before the encoding places the band limit on the video signal in this way to easily and properly control a produced code amount while preventing degradation of visual image quality. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image encoding device, an image encoding method, and a computer program, and is particularly suitable for use in encoding a moving image.

  Conventionally, as an image encoding method, an intra-frame encoding method such as Motion JPEG or Digital Video, or H.264 using inter-frame predictive encoding is used. 261, H.H. Coding schemes such as H.263, MPEG-1 and MPEG-2 are known. These encoding methods are internationally standardized by ISO (International Organization for Standardization) and ITU (International Telecommunication Union). The intra-frame coding method performs coding independently for each frame, and is easy to manage the frames, so that it is most suitable for an apparatus that requires editing of moving images and special reproduction. In addition, the inter-frame predictive coding method has a feature that coding efficiency is high because inter-frame prediction based on a difference in image data between frames is used.

  FIG. 16 shows an example of the configuration of a conventional moving picture encoding apparatus that performs encoding using MPEG-2, and FIG. 17 shows an example of the structure of a coded picture encoded by the moving picture encoding apparatus. As shown in FIG. 17, the structure of an encoded picture encoded by MPEG is configured by a combination of three types of pictures having different prediction methods. These three types of pictures are called an I picture (intraframe coding), a P picture (forward prediction coding), and a B picture (bidirectional prediction coding), respectively.

  A combination of a plurality of these pictures is called a group of picture (hereinafter referred to as GOP). Since this GOP includes only one I picture, it is often an editing unit. These prediction modes are switched by the switches 101 and 108 by a control signal from the mode determiner 105 as shown in FIG. In the I picture, DCT (Discrete Cosine Transform) is performed by the DCT unit 102 on the encoded image itself. On the other hand, in the P picture and the B picture, the DCT is performed by the DCT unit 102 on the error signal of the motion compensation prediction by the motion compensator 109.

After the quantization of the DCT coefficient obtained by the DCT unit 102 is performed by the quantizer 103 according to the control based on the output from the rate controller 100, the variable length coding is variable together with other additional information such as a motion vector. This is performed by the long encoder 104, and the code string is output as a “bit stream”. At this time, the quantization scale is controlled by the rate controller 100 according to the code amount from the variable length encoder 104. On the other hand, the output coefficient of the quantizer 103 is supplied to an inverse quantizer 106 and an inverse DCT unit 107, and is locally decoded and stored in a frame memory (FM) 110 for each block.
In the GOP structure, the interval between the I picture and the P picture is represented by M, and the number of frames in the GOP is represented by N. In the example shown in FIG. 17, M = 3 and N = 15.

  Since MPEG-2 performs variable length coding, the amount of generated code per unit time is not constant. Therefore, it is possible to control to a desired bit rate by appropriately changing the quantization scale at the time of quantization in the quantizer 103 in units of macroblocks. Step 3 of MPEG-2 Test Model 5 (refer to Non-Patent Document 1, hereinafter abbreviated as TM5) describes a method of changing the quantization scale based on the dispersion value of each macroblock.

  Here, the compression rate can be increased by increasing the quantization scale at the time of quantization. However, when the quantization scale is increased, the quantization error of the high frequency component increases, coding distortion called block distortion or mosquito distortion occurs, and the image quality significantly decreases. Further, since the interframe coding method uses interframe prediction, the coding is performed depending on both the high frequency component in the spatial direction and the amount of motion in the temporal direction. Therefore, even if a simple image having no high-frequency component moves rapidly, a large amount of high-frequency component is included in the prediction error, so that much coding distortion occurs.

In addition to this basic structure of video coding, the input image is preliminarily subjected to a band limiting filter to remove visually inconspicuous high frequency components and reduce the amount of code, thereby reducing the subjective image quality of the decoded image. It is generally performed to improve. At this time, the characteristics of the band limiting filter are set according to the compression rate, and are set to a narrow band when the compression rate is high, and to a wide band when the compression rate is low. Furthermore, a method is known in which the band limiting filter is controlled according to the amount of generated code that is an encoded output.
Other detailed contents such as MPEG will be entrusted to international standard documents by ISO / IEC.

MPEG-2 Test Model, Document ISO / IEC JTC1 SC29 WG11 / 93-400, Test Model Editing Committee, April 1993.

  However, in the conventional technique, when the generated code amount is large, the band is remarkably limited by the filter, and not only high-frequency components but also low-frequency component image information may be deleted. Since the low-frequency component has a great influence on the visual sense, if it is deleted, the visual image quality is significantly deteriorated.

  In addition, when a moving image having a sequence with a large amount of motion is encoded even if the high-frequency component of the moving image is small, the generated code amount increases, and the band limiting filter acts in a direction to remove the high-frequency component. For this reason, not only the high-frequency component in the moving part but also the stationary part with a small amount of motion is removed, and the resolution of the still part where the human visual resolution is high is deteriorated. Therefore, the overall image quality evaluation of the reproduced image is greatly reduced. At the same time, if the cut-off frequency of the band limiting filter fluctuates, the resolution of the still portion fluctuates, so that when the entire image is viewed, the result is an unnatural image.

As described above, the conventional technique cannot achieve high image quality while efficiently suppressing the code amount.
The present invention has been made in view of the above-described problems, and an object thereof is to efficiently encode a moving image so as to obtain a high-quality image.

  An image encoding apparatus according to the present invention includes a preprocessing unit that preprocesses input moving image data, an encoding unit that encodes moving image data preprocessed by the preprocessing unit, and the encoding unit. Storage means for storing encoded moving image data, first coefficient generating means for generating a first coefficient for preprocessing control using the feature amount of the input moving image data, and the code A second coefficient for preprocessing control is generated using the control parameter used when the moving image data is encoded by the converting unit and the generated code amount of the moving image data stored by the storing unit. Second coefficient generating means for pre-processing the input moving image data in accordance with the first coefficient and the second coefficient, and the encoding means , The feature amount of the input moving image data and It said storage means based on the generated code amount of the moving image data stored in, characterized by coding a moving picture data which has been preprocessed by the preprocessing means.

  The image encoding method of the present invention includes a preprocessing step for preprocessing input moving image data, an encoding step for encoding moving image data preprocessed by the preprocessing step, and the encoding step. An accumulation step of accumulating the encoded moving image data; a first coefficient generation step of generating a first coefficient for preprocessing control using the feature amount of the input moving image data; A second coefficient for preprocessing control is generated using the control parameter used when the moving image data is encoded by the converting step and the generated code amount of the moving image data stored by the storing step A second coefficient generation step, wherein the preprocessing step preprocesses the input moving image data according to the first coefficient and the second coefficient, and performs the encoding step. Encoding the moving image data preprocessed by the preprocessing step based on the feature amount of the input moving image data and the generated code amount of the moving image data stored in the buffer step. It is characterized by doing.

  The computer program of the present invention includes a preprocessing step for preprocessing input video data, an encoding step for encoding video data preprocessed by the preprocessing step, and encoding by the encoding step. An accumulation step for accumulating the moving image data, a first coefficient generation step for generating a first coefficient for preprocessing control using a feature amount of the input moving image data, and the encoding step The second coefficient for generating the second preprocessing control coefficient is generated by using the control parameter used when the moving image data is encoded by the above and the generated code amount of the moving image data accumulated by the accumulation step. The coefficient generation step, and the preprocessing step preprocesses the input moving image data according to the first coefficient and the second coefficient, and The encoding step encodes the moving image data preprocessed by the preprocessing step based on the feature amount of the input moving image data and the generated code amount of the moving image data accumulated in the buffer step. It is characterized by becoming.

  According to the present invention, the first coefficient for preprocessing control generated using the feature amount of the input moving image data, the control parameter used in encoding the moving image data, and the encoded Pre-processing the input moving image data according to the second coefficient for preprocessing control generated using the generated code amount of the moving image data, and encoding the preprocessed moving image data Accordingly, it is possible to easily control the generated code amount while interlocking the preprocessing and the code amount control to prevent the image quality from being visually deteriorated. Like that.

(First embodiment)
Next, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating an example of a configuration of an image encoding device according to the present embodiment. In the image coding apparatus according to the present embodiment, the minimum control unit is a macro block.
In FIG. 1, reference numeral 1 denotes a feature amount measuring device that measures a variance for each macroblock as a feature amount of an input image. Reference numeral 2 denotes a feature amount memory that accumulates feature amounts from the feature amount measuring device 1. In the present embodiment, it is assumed that a feature amount for one frame is accumulated.

A filter coefficient calculator 3 calculates a filter coefficient from the accumulated feature amount. Reference numeral 4 denotes a filter controller that determines a control value of the filter.
Reference numeral 5 denotes a filter executed before encoding. In the present embodiment, the filter 5 is a two-dimensional filter of 3 pixels × 3 lines. The filter 5 is a low-pass filter that sets a band based on a variable k, as shown in the following (formula 1). When the minimum value of the variable k is 2, and the value increases, the strength of the filter decreases and the band to be limited becomes narrow.

Reference numeral 6 denotes a frame memory for accumulating the image signal output from the filter 5. In the present embodiment, similarly to the feature amount memory 2, an image signal for one frame is stored. Reference numeral 7 denotes an encoder that encodes an image signal stored in the frame memory 6.
Reference numeral 8 denotes an encoder output buffer for accumulating the stream output from the encoder 7. The buffer 8 outputs a generated code amount for each frame based on the stream output from the encoder 7 and a buffer pointer assuming a decoder buffer. The decoder buffer pointer is a value obtained by subtracting the encoder buffer pointer from the buffer size.

  A code amount controller 9 controls the encoder 7 from the feature amount stored in the feature amount memory 2 and the generated code amount stored in the buffer 8. A filter coefficient calculator 10 calculates the coefficient of the filter 5 from the control parameter output to the encoder 7 and the decoder buffer pointer output from the buffer 8. In this embodiment, the control parameter is a quantization scale. Reference numeral 11 denotes a coefficient memory that accumulates the coefficients output from the filter coefficient calculator 10. In the present embodiment, as in the feature amount memory 2, one frame of coefficients is accumulated.

Next, the operation of the image coding apparatus of the present embodiment having the above configuration will be described.
Processing in this embodiment is roughly classified into two. This is processing of the first frame (hereinafter abbreviated as the first frame) and processing of other frames. The process for the first frame is executed until an image that has been subjected to filter processing for one frame is prepared before encoding. On the other hand, the other normal processing includes a process of encoding the frame subjected to the filter and a process of applying a filter to a frame to be encoded next.

Next, an example of the operation of the image coding apparatus when performing the filtering process for the first frame will be described with reference to the flowchart of FIG.
First, the video signal is input to the feature amount measuring device 1, and the feature amount measuring device 1 calculates a feature amount for one frame (step S10). In the present embodiment, the feature quantity measuring device 1 calculates a feature quantity for each macroblock. Here, the feature amount calculated by the feature amount measuring device 1 is referred to as act. The feature value act is a value obtained by adding 1 to the minimum value of the variance values of the four blocks in the luminance signal in the macroblock. The calculation formula is shown in the following (Formula 2).

  Here, Pj is a luminance pixel value in the block. P_avg is an average value of the luminance pixel values in the block. MIN_block0to3 indicates that the minimum value is selected from the dispersion values of four blocks.

Next, the calculated feature quantity act is stored in the feature quantity memory 2 for one frame (step S11). At this time, an average value avg_act in the frame of the feature amount act is calculated and stored in the feature amount memory 2.
Next, the filter coefficient calculator 3 calculates a filter coefficient ka for each macroblock from the accumulated feature amount (step S12).
FIG. 3 is a graph showing an example of the relationship between the feature amount act and the filter coefficient ka. In FIG. 3, actA in the graph is the minimum value of the feature quantity act, and is a value that the curve is asymptotic. In this embodiment, the initial value is zero.
When the feature amount act is greater than or equal to actA, the filter coefficient Ka decreases.
Next, a filter is executed on the first frame using the filter coefficient ka and stored in the frame memory 6 (step S13).

Next, an example of the operation of the image encoding device when performing encoding processing in a steady state will be described with reference to the flowchart of FIG. When encoding a moving image, this processing, which is a processing unit of one frame, is repeated.
First, the code amount controller 9 calculates a quantization scale mquant according to TM5 based on the generated code amount from the buffer 8. Next, for this quantization scale mquant, the feature value act for each frame in the feature value memory 2 accumulated and the average value avg_act in the frame of the feature value act are calculated, and the following (Equation 3) and Weighting is performed according to (Expression 4).

QP = mquant × w (3 formulas)
w = (avg_act-act) / avg_act (4 formulas)
The final quantization scale (code amount control value) QP is calculated by the above processing, and the encoder 7 is controlled (step S31).
Next, the encoder 7 encodes the frame stored in the frame memory 6 (step S32).
Next, the filter coefficient ke is calculated based on the quantization scale QP and the buffer pointer from the buffer 8 (step S33). In this embodiment, a filter coefficient is obtained from each of the quantization scale QP and the buffer pointer, and one of the two is selected as the final filter coefficient ke.
Next, the filter coefficient ke is stored in the coefficient memory 11 (step S34).

An example of the operation of the image encoding device (operation of step S33) when obtaining the filter coefficient ke will be described with reference to the flowchart of FIG.
First, the filter coefficient kq is calculated from the quantization scale QP (step S40). FIG. 6 is a graph showing an example of the relationship between the quantization scale QP and the filter coefficient kq. In FIG. 6, QPA in the graph is the value of the quantization scale QP when the filter strength is the lowest, and is a value that the curve is asymptotic.
In the present embodiment, the filter coefficient kq decreases when the value of the quantization scale QP is equal to or greater than this QPA.

Next, the filter coefficient kb is calculated from the decoder buffer pointer DBP (step S41). FIG. 7 is a graph showing an example of the relationship between the decoder buffer pointer DBP and the filter coefficient kb. In FIG. 7, DB in the graph is the buffer size, and the value that the curve is asymptotic is DB / 2.
In order to prevent underflow of the decoder buffer, the filter coefficient kb decreases when the decoder buffer pointer DBP becomes ½ or less of the buffer size.

  Next, the calculated filter coefficients kq and kb are compared, and the larger value is set as the filter coefficient ke in the filter coefficient calculator 10 (steps S42, S43, and S44). The filter coefficient ke is stored in the coefficient memory 11.

Next, an example of the operation of the image coding apparatus when performing the filtering process (stationary frame filter process) at the normal time will be described with reference to the flowchart of FIG. In encoding a moving image, this processing is repeated for each frame processing unit. In FIG. 8, steps S10 to S13 are equivalent to steps S10 to S13 of FIG.
In step S12, a filter control value k is calculated based on the filter coefficient ka obtained from the feature quantity and the filter coefficient ke stored in the coefficient memory 11 (step S30).

Next, an example of processing in the filter controller 4 will be described with reference to the flowchart of FIG. In FIG. 9, step S12 is equivalent to step S12 of FIG.
When the filter coefficient ka is calculated in step S12, the accumulated filter coefficient ke is input from the coefficient memory 11 (step S51).

Next, it is determined whether or not the filter coefficient ka is larger than the filter coefficient ke. If it is larger, the filter coefficient ka is set as the filter control value k. On the other hand, when the filter coefficient ka is not larger than the filter coefficient ke, the filter coefficient ke is set as the filter control value k (steps S52 to S54).
Note that the filter control value k may be calculated from the filter coefficients ke and ka by the following (formula 5).
k = (XA × ka + XE × Ke) / (XA + XE) (Formula 5)
Here, XA is a weighting constant for the filter coefficient ka, and XE is a weighting constant for the filter coefficient ke.

  As described above, in the present embodiment, the feature quantity of the moving image is measured, the filter coefficient ke obtained based on the quantization scale QP from the buffer pointer and the code quantity control, and the filter coefficient ka obtained from the feature quantity. Is used to obtain the filter control value k, and based on the obtained filter control value k, the band of the video signal is limited by the filter 5 immediately before encoding, thereby preventing deterioration of visual image quality. However, the generated code amount can be easily and suitably controlled.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the description of the present embodiment, the same parts as those in the first embodiment described above are denoted by the same reference numerals as those in FIGS.

  In the first embodiment described above, the feature amount memory 2 stores the feature amount for one frame, the frame memory 6 stores the image signal for one frame, and the coefficient memory 11 stores the coefficient for one frame. I did it. In contrast, in the present embodiment, the feature amount memory 2 accumulates feature amounts for M frames, the frame memory 6 stores image signals for M frames, and the coefficient memory 11 accumulates coefficients for M frames. I am doing so. As described above, in the first embodiment, the filter process, the accumulation process, and the encoding process are performed for each frame. However, in the present embodiment, these processes are performed for each M frame. ing.

  The configuration of the image encoding device of the present embodiment is the same as that of the image encoding device of the first embodiment shown in FIG. 1, but as described above, the feature amount memory 2 stores the feature amounts for M frames. The frame memory 6 stores image signals for M frames, and the coefficient memory 11 stores coefficients for M frames.

Next, the operation of the image coding apparatus according to this embodiment will be described.
Processing in this embodiment is roughly classified into two. The process from the top to the M frame (hereinafter abbreviated as the top M frame) and the other frames. The process for the first M frame is executed until an image that has been subjected to filter processing for M frames is prepared before encoding. On the other hand, the other normal processing includes processing for encoding the frame subjected to the filter and processing for filtering the frame to be encoded next. Here, the M frame is an MPEG ordering unit and is equal to the memory area essential for an encoder that handles B pictures.

  The operation of the image encoding apparatus when performing the filtering process for the top M frame is the same as that of the flowchart of FIG. 2 described above except that the operation for M frames is performed. That is, in step S10, the feature quantity measuring device 1 calculates a feature quantity act for M frames of the video signal for each macroblock. The calculation formula of the feature amount act is as described above (Formula 2).

Next, in step S <b> 11, the calculated feature quantity act is stored in the feature quantity memory 2 for M frames. At this time, an average value avg_act in the frame of the feature amount act is calculated and stored in the feature amount memory 2.
Next, in step S12, an average value actm (hereinafter referred to as an M frame average feature amount) of the feature amount in the macro block at the same position in the M frame is calculated from the accumulated feature amount act for each macro block. Further, a filter coefficient ka is calculated from the M frame average feature value actm.

The relationship between the M frame average feature value actm and the filter coefficient ka is the same as that shown in FIG. When the value of the M frame average feature value actm is greater than or equal to actA, the filter coefficient ka decreases and the filter strength increases.
Next, in step S13, using the filter coefficient ka, a filter is executed on the first M frames and stored in the frame memory.

  The operation of the image encoding apparatus when performing the regular encoding process is the same as the flowchart of FIG. 4 described above except that the operation for M frames is performed. That is, in step S31, the code amount controller 9 calculates the quantization scale mquant, and the accumulated feature amount act for each frame in the feature amount memory 2 and the feature amount for the calculated quantization scale mquant. The average value avg_act in the frame of act is calculated, and weighted according to the above (Expression 3) and (Expression 4). The final quantization scale (code amount control value) QP is calculated by the above processing, and the encoder 7 is controlled.

  Next, in step S32, the encoder 7 encodes the frame stored in the frame memory 6, and in step S33, the filter coefficient ke is calculated based on the quantization scale QP and the buffer pointer from the buffer 8. In step S34, the filter coefficient ke is stored in the coefficient memory 11. In the present embodiment, the filter coefficient is obtained from each of the quantization scale QP and the buffer pointer, and one of the two is selected as the final filter coefficient ke.

Further, the operation of the image coding apparatus when obtaining the filter coefficient ke is as shown in the flowchart of FIG.
The operation of the image encoding device when performing the regular filter processing is the same as that of the flowchart of FIG. 8 described above except that the operation for M frames is performed. That is, in step S10, feature values for M frames are input, feature values for these M frames are accumulated in step S11, and filter coefficients for each macroblock are calculated from the feature values for these M frames in step S12. The average value of Ka is obtained.

  In step S30, a filter control value k is calculated based on the obtained average value of the filter coefficients ka and the filter coefficient ke. In step S13, filter processing is executed using the calculated filter control value k. The processed video signal is stored in the frame memory 6.

Further, the processing in the filter controller 4 is the same as the flowchart of FIG. 9 described above except that the operation for M frames is performed. That is, in step S12, the average value of the filter coefficient Ka for each macroblock is calculated from the feature amounts for M frames, and the accumulated filter coefficient ke is input from the coefficient memory 11 in step S51. At this time, an average value kem (hereinafter referred to as M frame average filter coefficient) of the filter coefficient ke in the macroblock at the same position in the M frame is calculated from the filter coefficient ke.
Next, the filter control value k is calculated using the M frame average filter coefficient kem and the average value of the filter coefficient ka (steps S52 to S54). At this time, the filter control value k may be calculated using the above-described (Formula 5).

As described above, in the present embodiment, the filters having the same strength are applied to a plurality of frames stored in the memory. Therefore, in addition to the effect of the first embodiment described above, a sharp band limitation of an image is performed. Can be prevented, and deterioration of visual image quality can be more reliably prevented.
Further, since the filtering process and the encoding process can be performed separately by the arbitration of the memory, the two processes can be operated in parallel.

(Third embodiment)
Next, a third embodiment of the present invention will be described. In the description of the present embodiment, the same parts as those in the first embodiment described above are denoted by the same reference numerals as those in FIGS.

FIG. 10 is a block diagram illustrating an example of the configuration of the image encoding device according to the present embodiment. In the image coding apparatus of this embodiment, the minimum control unit is a macro block.
In FIG. 10, reference numeral 21 denotes a feature amount measuring device that measures a feature amount for each macroblock as a feature amount of an input image. In the present embodiment, the maximum value of the variance and inter-pixel difference for each macroblock is measured. Reference numeral 22 denotes a feature amount memory that accumulates feature amounts from the feature amount measuring device 11. In the present embodiment, feature amounts for M frames are accumulated.

  A filter coefficient calculator 23 calculates a filter coefficient from the accumulated feature amount. Reference numeral 24 denotes an ε filter that protects and smoothes edges. The ε filter 24 used in the present embodiment performs filter processing represented by the following (formula 6) and (formula 7).

  In the filter processing represented by (Expression 6) and (Expression 7), the difference value between the central pixel f (x, y) and its neighboring pixel f (x + i, y + j) is calculated, and the absolute value thereof is calculated. If the value is larger than ε, smoothing processing is performed by replacing the value of the neighboring pixel with the value of the central pixel (x, y = −M,..., 0, 1,..., M). . As a result, in the output g (x, y) of the ε filter 4, the edge component can be protected and noise that is a harmonic having a small amplitude can be smoothed and suppressed. Details of the ε filter are described in “Hiroshima Hara et al.,“ Ε-Separation Nonlinear Digital Filter and Its Applications ”, IEICE 57-146 [A-36] April 1982”.

A frame memory 25 stores the result of the filter processing in the ε filter 4.
Reference numeral 26 denotes a horizontal and vertical two-dimensional band limiting filter. The two-dimensional band limiting filter 26 used in the present embodiment is represented by the above (1), and is a two-dimensional filter of 3 pixels × 3 lines. Hereinafter, this two-dimensional band limiting filter is referred to as a k filter.

A frame memory 27 stores the result of the filter processing in the k filter 26. In the present embodiment, similarly to the feature amount memory 2, image signals for M frames are stored. Reference numeral 28 denotes an encoder that encodes an image signal stored in the frame memory 27.
Reference numeral 29 denotes an encoder output buffer for accumulating the stream output from the encoder 28. The buffer 29 outputs a generated code amount for each frame based on the stream output from the encoder 28 and a decoder buffer pointer assuming a decoder buffer. The decoder buffer pointer is a value obtained by subtracting the encoder buffer pointer from the buffer size.

  Reference numeral 30 denotes a code amount controller for controlling the encoder 8 based on the generated code amount output from the buffer 29. A filter coefficient calculator 31 calculates the coefficients of the ε filter 24 and the k filter 26 from the control parameter from the encoder 8 and the decoder buffer pointer output from the buffer 9. In this embodiment, the control parameter is a quantization scale.

  A coefficient memory 32 stores the coefficients calculated by the filter coefficient calculator 31. In the present embodiment, as in the feature amount memory 22, coefficients for M frames are accumulated. Reference numeral 33 denotes a k filter controller that controls the k filter 26. Reference numeral 34 denotes an ε filter controller that controls the ε filter 24.

Next, the operation of the image coding apparatus of the present embodiment having the above configuration will be described.
Processing in this embodiment is roughly classified into two. This is processing of the first M frame and processing of other frames. The processing for the first M frame is executed until an image that has been subjected to filter processing for M frames is prepared before encoding. On the other hand, the other normal processing includes a process of encoding the frame subjected to the filter and a process of applying a filter to a frame to be encoded next. Here, the M frame is an MPEG ordering unit and is equal to the memory area essential for an encoder that handles B pictures.

Next, an example of the operation of the image coding apparatus when performing the filtering process of the leading M frame (leading frame filtering process) will be described with reference to the flowchart of FIG.
First, the video signal is input to the feature amount measuring device 21, and the feature amount measuring device 21 calculates the maximum value md of the difference between adjacent pixels in the macroblock for each macroblock (step S110). Here, the maximum value md of the difference between adjacent pixels in the macroblock is a value obtained for the luminance signal (= 16 pixels × 16 lines) in the macroblock. The calculation formula of the maximum value md of the inter-pixel difference is shown in the following (Equation 8).

md = MAX_i0_15_j0_14 (abs (p (i + 1, j) -p (i, j))) (Expression 8)
Here, i is the horizontal pixel position in the macroblock. j is the vertical line position in the macroblock. p (i, j) is a pixel in the macroblock (coordinates (i, j)). abs indicates an absolute value. MAX_i0_15_j0_14 indicates that the maximum value in i = 0 to 14, j = 0 to 15 is selected.

Next, the calculated maximum value md of the inter-pixel difference is stored in the feature amount memory 22 for M frames (step S111).
Next, the ε filter controller 34 calculates the average value mdm in the macroblock at the same position in the M frame from the maximum value md of the inter-pixel difference, and further calculates the filter control value ε from the average value mdm (step). S112). In the present embodiment, it is assumed that the filter control value ε is calculated by the following conversion formula (formula 9).
ε = mdm / 2 (9 formulas)

  The ε filter 24 performs filter processing on the input video signal (input image) using the filter control value ε. The output image that has been filtered by the ε filter 24 is stored in the frame memory 25. In the first M frame, the filter process by the k filter 26 is not executed, and the output image in the frame memory 25 is accumulated in the frame memory 27 (step S113).

The operation of the image coding apparatus when performing a regular coding process is the same as that of the second embodiment described above (see FIG. 4). That is, in step S 31, the code amount controller 30 calculates the quantization scale QP from the generated code amount from the buffer 29 and controls the encoder 28.
Next, in step S32, the encoder 28 encodes the frame stored in the frame memory 27. In step S33, the coefficient calculator 31 receives the buffer pointer from the buffer 29 and the quantum amount from the code amount controller 30. The filter coefficient ke is calculated based on the quantization scale QP, and the filter coefficient ke is stored in the coefficient memory 32 in step S34. Note that in the encoding of moving images, this processing is repeated for each processing unit of M frames.

  Further, the operation of the image coding apparatus when obtaining the filter coefficient ke is as shown in the flowchart of FIG.

  Next, an example of the operation of the image coding apparatus when performing the filtering process (stationary frame filter process) at the normal time will be described with reference to the flowchart of FIG. Note that in the encoding of moving images, this processing is repeated for each processing unit of M frames. In FIG. 12, step S12 is equivalent to FIG.

  First, the video signal is input to the feature amount measuring device 21, and the feature amount measuring device 21 calculates the maximum value of the variance and inter-pixel difference for each macroblock (step S120). Since the processing regarding the maximum value md of the inter-pixel difference is as described above, a detailed description thereof will be omitted, and here, an operation until obtaining a variance value which is another feature amount will be described.

  In the present embodiment, the feature value measuring device 21 calculates a variance value for each macroblock. The calculated variance value for each macroblock is referred to as act. The variance value act is a value obtained by adding 1 to the minimum value of the variance values of four blocks in the luminance signal in the macroblock. The calculation formula of the dispersion value act is as shown in the above (Formula 2).

Next, the calculated variance value act is stored in the feature amount memory 22 for M frames simultaneously with the maximum value md of the inter-pixel difference (step S121).
Next, as described in step S12, the ε filter controller 34 converts the maximum value md of the inter-pixel difference into a filter control value (threshold value) ε in the ε filter 24, and uses this filter control value ε. The ε filter 24 performs a filtering process on the input image and accumulates it in the frame memory 25 (step S122).

Next, the k filter controller 33 calculates a filter control value k based on the variance value act for each macroblock accumulated in the feature amount memory 22 and the accumulated filter coefficient ke in the coefficient memory 32. (Step S123).
Next, using the calculated filter control value k, the k filter 6 performs filter processing on the image signal accumulated in the frame memory 25 and accumulates it in the frame memory 27 (step S124).

Here, an example of processing in the ε filter controller 34 will be described with reference to the flowchart of FIG. 13.
First, the filter coefficient ke is input from the coefficient memory 12 (step S130). Next, the filter coefficient ke is compared with a predetermined value KE (step S131). If the filter coefficient ke is smaller than the predetermined value KE, the ε filter 24 is turned off (step S132). ). On the other hand, if the filter coefficient ke is equal to or greater than a predetermined value KE, the process of step S12 described above is executed to calculate the filter control value ε, and the ε filter 24 is turned on (step S133).

Next, an example of processing in the k filter controller 33 will be described with reference to the flowchart of FIG.
First, an average value actm (hereinafter referred to as M frame average variance) in a macroblock at the same position in M frames is calculated from the variance value act for each macroblock stored in the feature amount memory 22, and a filter coefficient ka is calculated. Calculate (step S140).
The relationship between the M frame average variance actm and the filter coefficient ka is the same as that shown in FIG. ActA in the graph is the minimum value of the M frame average variance actm, and is a value that the curve is asymptotic. In this embodiment, the initial value is zero. When the M frame average variance actm is greater than or equal to actA, the filter coefficient ka decreases and the filter strength increases.

Next, the filter coefficient ke accumulated from the coefficient memory 32 is input (step S141). At this time, the average value kem (M frame average filter coefficient) of the macroblock at the same position in the M frame is calculated from the filter coefficient ke.
Next, using the M frame average filter coefficient kem obtained from the coefficient memory 32 and the filter coefficient ka obtained from the coefficient calculator 23, the filter control value k is calculated according to the above (Equation 7) (step S142). .

  As described above, in this embodiment, two types of filters immediately before encoding are provided, and by controlling each of them simultaneously, noise in the flat portion is removed by one filter, and band limitation is performed on the other side. Therefore, in addition to the effects in the first and second embodiments described above, it is possible to more suitably control the amount of generated code while more surely preventing deterioration in visual image quality.

In each of the above-described embodiments, the variance value act is obtained using (Equation 6) as the image feature amount based on the variance value in the block. However, other equations that can assume the generated code amount are used. Thus, the image feature amount may be obtained.
In addition, a two-dimensional k filter as shown in (Expression 3) is used as the band limiting filter, but another type of filter may be used. At this time, if there are a plurality of coefficients for controlling the band of the filter, a table in which a combination of a plurality of coefficients is defined for k in this embodiment may be prepared and used.

Further, when the filter coefficient is obtained from the buffer pointer, the effect of the filter is set to ½ or less of the buffer size, but other values may be used.
Further, the relational expression for calculating the filter coefficient may be an expression other than the expression described in the present embodiment. A simple proportional expression may be used for easier implementation.
Furthermore, although the cycle for switching the filters is set to the macroblock unit, the switching may be performed in units other than this.
Furthermore, in the third embodiment, either one of the two types of filters may be fixedly handled in order to simplify the processing. For example, the filter coefficient (threshold value) ε of the ε filter may be a fixed value.

(Fourth embodiment)
The image encoding apparatus in the first to third embodiments described above can be realized using a computer as shown in FIG.
In FIG. 15, reference numeral 1000 denotes a central processing unit (CPU) that controls the entire computer and performs various processes. Reference numeral 1001 denotes an operating system (OS), software, data, and storage area necessary for control of the computer. Is a memory that provides The memory 1001 is also used as a work area when the CPU 1000 performs various processes.

A bus 1002 connects various devices and exchanges data and control signals. Reference numeral 1003 denotes a storage device (recording medium) that stores various types of software. Reference numeral 1004 denotes a storage device (recording medium) that accumulates moving image data. Reference numeral 1005 denotes a monitor that displays images and system messages from the computer.
A communication interface 1007 transmits encoded data to the communication circuit 1008, and is connected to a LAN, a public line, a wireless line, a broadcast wave, and the like outside the apparatus. Reference numeral 1006 denotes a terminal for starting up the computer and setting various conditions such as a bit rate.

  The memory 1001 stores an OS for controlling the entire computer and operating various software and software to be operated, an area for reading image data for encoding, and temporarily stores code data. There is a working area for storing a code area to be used, parameters for various operations, and the like.

  In the computer having such a configuration, prior to processing, based on the operation of the terminal 1006 by the user, the moving image data to be encoded is selected from the moving image data stored in the storage device 1004 and the computer is instructed to start. The Then, the software stored in the storage device 1003 is expanded in the memory 1001 via the bus 1002, and the software is activated.

Then, the encoding operation of the moving image data stored in the storage device 1004 by the CPU 1000 is performed by the program code (the above-mentioned software) according to the flowcharts shown in FIGS. 4, 5, 8, and 9 and FIGS. Will be executed.
As described above, the computer according to the present embodiment functions as a device that implements the image coding device according to the first to third embodiments.

(Other embodiments of the present invention)
Software program code for realizing the functions of the above-described embodiment for a computer in an apparatus or a system connected to the various devices so as to operate the various devices to realize the functions of the above-described embodiments. Are implemented by operating the various devices in accordance with a program stored in a computer (CPU or MPU) of the system or apparatus.

  In this case, the program code of the software itself realizes the functions of the above-described embodiments, and the program code itself and means for supplying the program code to the computer, for example, the program code The stored recording medium constitutes the present invention. As a recording medium for storing the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code supplied by the computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) or other application software in which the program code is running on the computer, etc. It goes without saying that the program code is also included in the embodiment of the present invention even when the functions of the above-described embodiment are realized in cooperation with the embodiment.

  Further, after the supplied program code is stored in the memory provided in the function expansion board of the computer or the function expansion unit connected to the computer, the CPU provided in the function expansion board or function expansion unit based on the instruction of the program code Needless to say, the present invention also includes a case where the functions of the above-described embodiment are realized by performing part or all of the actual processing.

1 is a block diagram illustrating an example of a configuration of an image encoding device according to a first embodiment of this invention. FIG. 5 is a flowchart illustrating an example of an operation of the image encoding device when performing the filtering process of the first frame according to the first embodiment of this invention. It is the figure which showed the 1st Embodiment of this invention and graphed and showed an example of the relationship between a feature-value and a filter coefficient. It is a flowchart which shows the 1st Embodiment of this invention and demonstrates an example of operation | movement of the image coding apparatus at the time of performing the encoding process at the time of steady state. It is a flowchart which shows the 1st Embodiment of this invention and demonstrates an example of operation | movement of the image coding apparatus at the time of calculating | requiring a filter coefficient. It is the figure which showed the 1st Embodiment of this invention and graphed and showed an example of the relationship between a quantization scale and a filter coefficient. It is the figure which showed the 1st Embodiment of this invention and graphed and showed an example of the relationship between a buffer pointer and a filter coefficient. 5 is a flowchart illustrating an example of an operation of the image encoding device when performing a filtering process in a steady state according to the first embodiment of this invention. It is a flowchart which shows the 1st Embodiment of this invention and demonstrates an example in the process in a filter controller. It is a block diagram which shows the 3rd Embodiment of this invention and shows an example of a structure of an image coding apparatus. 10 is a flowchart illustrating an example of the operation of the image encoding device when performing the filtering process for the first M frames according to the third embodiment of this invention. It is a flowchart which shows the 3rd Embodiment of this invention and demonstrates an example of operation | movement of the image coding apparatus at the time of performing the filter process at the time of steady state. It is a flowchart which shows the 3rd Embodiment of this invention and demonstrates an example in the process in an epsilon filter controller. It is a flowchart which shows the 3rd Embodiment of this invention and demonstrates an example of the process in a k filter controller. It is the block diagram which showed the 4th Embodiment of this invention and showed an example of the structure of the computer system which implement | achieves an image coding apparatus. It is the block diagram which showed the prior art and showed an example of the structure of the conventional moving image encoder. It is the figure which showed the prior art and showed an example of the structure of the encoded picture.

Explanation of symbols

1, 21 Feature amount measuring device 2, 22 Feature amount memory 3, 10, 23, 31 Coefficient calculator 4 Filter controller 5 Filter 6, 25, 27 Frame memory 7, 28 Encoder 8, 29 Buffer 9, 30 Code Quantity controller 11, 32 Coefficient memory 24 ε filter 26 k filter 33 k filter controller 34 ε filter controller 1000 CPU
1001 Memory 1002 Bus 1003, 1004 Storage device (recording medium)
1005 Monitor 1006 Terminal 1007 Communication I / F

Claims (11)

Preprocessing means for preprocessing input video data;
Encoding means for encoding moving image data preprocessed by the preprocessing means;
Storage means for storing moving image data encoded by the encoding means;
First coefficient generation means for generating a first coefficient for preprocessing control using the feature amount of the input moving image data;
A second coefficient for preprocessing control using the control parameter used when the moving image data is encoded by the encoding unit and the generated code amount of the moving image data stored by the storage unit Second coefficient generation means for generating
The preprocessing means preprocesses the input moving image data according to the first coefficient and the second coefficient,
The encoding means is the moving image data preprocessed by the preprocessing means based on the feature amount of the input moving image data and the generated code amount of the moving image data stored in the storage means. An image encoding apparatus that encodes   The image coding apparatus according to claim 1, further comprising a feature amount measuring unit that measures the feature amount of the input moving image data for each region.   The feature amount measuring unit divides the input moving image data in an area equivalent to a unit encoded by the encoding unit, and distributes the input moving image data for each of the divided regions. The image coding apparatus according to claim 2, wherein a value is measured.   The encoding unit calculates a quantization scale based on a generated code amount of the moving image data stored in the storage unit, and calculates the calculated quantization scale as a feature amount of the input moving image data. 4. The image encoding device according to claim 1, wherein the pre-processed moving image data is encoded using the weighted quantization scale as the control parameter. 5. .   The second coefficient generation unit generates a coefficient based on a quantization scale as the control parameter and a coefficient based on a generated code amount of moving image data accumulated by the accumulation unit, and generates the generated two coefficients. The image coding apparatus according to claim 1, wherein the second coefficient is obtained based on the comparison result. Feature quantity storage means for storing feature quantities for a plurality of frames of the input moving image data;
Image storage means for storing moving image data for a plurality of frames preprocessed by the preprocessing means;
6. The image according to claim 1, further comprising a second coefficient accumulation unit that accumulates second coefficients for a plurality of frames calculated by the second coefficient calculation unit. Encoding device. The preprocessing means is a first preprocessing means for protecting edges of the input moving image data and smoothing the input moving image data based on a feature amount of the input moving image data. When,
Based on the first coefficient generated by the first coefficient generation means and the second coefficient generated by the second coefficient generation means, a band limitation of the input moving image data is performed. The image coding apparatus according to claim 1, further comprising: two preprocessing units. The feature amount measuring unit divides the input moving image data in an area equivalent to a unit encoded by the encoding unit, and distributes the input moving image data for each of the divided regions. A dispersion value measuring means for measuring a value;
8. The image encoding apparatus according to claim 7, further comprising difference value measuring means for measuring a difference value between adjacent pixels of the input moving image data for each of the divided areas. The first preprocessing means smoothes only a flat portion of the input moving image data,
The image coding apparatus according to claim 7 or 8, wherein the second preprocessing means band-limits the spatial frequency of the input moving image data in a two-dimensional space. A preprocessing step for preprocessing input video data;
An encoding step for encoding the moving image data preprocessed by the preprocessing step;
An accumulation step for accumulating moving image data encoded by the encoding step;
A first coefficient generation step of generating a first coefficient for preprocessing control using the feature amount of the input moving image data;
A second coefficient for preprocessing control using the control parameter used when moving image data is encoded by the encoding step and the generated code amount of the moving image data accumulated by the accumulation step A second coefficient generation step for generating
The preprocessing step preprocesses the input moving image data according to the first coefficient and the second coefficient,
In the encoding step, the moving image data preprocessed by the preprocessing step based on the feature amount of the input moving image data and the generated code amount of the moving image data stored in the buffer step An image encoding method characterized by encoding. A preprocessing step for preprocessing input video data;
An encoding step for encoding the moving image data preprocessed by the preprocessing step;
An accumulation step for accumulating moving image data encoded by the encoding step;
A first coefficient generation step of generating a first coefficient for preprocessing control using the feature amount of the input moving image data;
A second coefficient for preprocessing control using the control parameter used when moving image data is encoded by the encoding step and the generated code amount of the moving image data accumulated by the accumulation step Causing the computer to execute a second coefficient generation step of generating
The preprocessing step preprocesses the input moving image data according to the first coefficient and the second coefficient,
In the encoding step, the moving image data preprocessed by the preprocessing step based on the feature amount of the input moving image data and the generated code amount of the moving image data stored in the buffer step The computer program characterized by encoding.

Images