JP4247089B2 - Non-interlaced image processing apparatus and processing method - Google Patents

Description

  The present invention relates to a moving image processing apparatus, and more particularly to an image processing apparatus and processing method for processing a non-interlaced image obtained from a series of a plurality of interlaced images.

  In recent years, JEPPG2000 is known as a compression encoding method suitable for handling high-definition images. There is also a standard called Motion JPEG 2000 that displays a moving image by continuously playing still images encoded in the JPEG 2000 format.

In a series of interlaced images taken by a video camera or the like, there is a parameter that does not exist in a still image, that is, the amount of movement (change amount) of a subject between frames in comparison with the interlaced image of the previous frame. Several moving image processing apparatuses that detect the moving speed of a subject in a frame using the parameter of the moving amount and adaptively perform compression coding processing have already been proposed (see, for example, Patent Document 1). ).
Japanese Patent Publication No. 04-77517

  In the moving image processing apparatus, a calculation process for obtaining a difference in image data of a subject between frames and obtaining a moving speed based on the obtained difference data is performed. For this reason, the amount of data to be processed is large, time is required for calculation, and a large amount of memory is required.

  The present invention determines the moving speed of the subject by using a small amount of data and a simple calculation process without using the difference of the subject image data between the frames, and based on the determination result. An object of the present invention is to provide an image processing apparatus and an image processing method for non-interlaced images to be processed.

The non-interlaced image processing device according to claim 1 is a non -interlaced image processing device (10) formed by alternately arranging two consecutively scanned interlaced images for each scanned line, A wavelet transform unit (step S4), a determination unit (step S11), a data deletion processing unit (step S6), and an encoding processing unit (step S8). The interlaced image data is subjected to level 1 or higher two-dimensional discrete wavelet transform, and the determining means includes the wavelet coefficients of the subbands at level 1 among the wavelet coefficients obtained by the wavelet transform means, respectively. A code consisting of a pixel matrix smaller than the subband Is divided into blocks, each code block, the sum of the absolute values of the coefficient values of the code blocks of 1LH, divided by the sum of the absolute values of the coefficient values of the code blocks 1HL positions corresponding to the code block ( In step S24), it is determined that the subject is moving when it is larger than the predetermined first threshold value (step S25), and it is determined that the subject is not moving if it is equal to or smaller than the first threshold value. When the value determined based on the sum of the divided values of the code blocks determined to be moving (step S30) is greater than a predetermined second threshold value, It is determined that the moving speed is high (step S31), and it is determined that the moving speed of the subject in the image is low when it is equal to or less than the second threshold value. With respect to an image for which the moving speed of the subject is determined to be high by the determination means, data deletion processing is executed in which the data deletion amount of the 1LH subband is smaller than that of the image determined to be low. And the encoding processing means executes encoding processing using data based on the wavelet coefficients subjected to data deletion processing by the data deletion processing means .
The non-interlaced image processing apparatus according to claim 2, wherein the determination unit adds the sum of the divided values of the code blocks determined to be moving to the code block determined to be moving. When the value obtained by dividing by the total number is larger than the predetermined second threshold value (step S30), it is determined that the moving speed of the subject in the image is high (step S31), and the second threshold value is reached. The non-interlaced image processing apparatus according to claim 1, wherein the moving speed of the subject in the image is determined to be low when the value is equal to or less than the value .

The non-interlaced image processing device according to claim 3 is a non-interlaced image processing device (10) formed by alternately arranging two interlaced images scanned successively for each scanned line, A wavelet transform unit (step S4), a determination unit (step S11), a data deletion processing unit (step S6), and an encoding processing unit (step S8). The interlaced image data is subjected to level 1 or higher two-dimensional discrete wavelet transform, and the determining means includes the wavelet coefficients of the subbands at level 1 among the wavelet coefficients obtained by the wavelet transform means, respectively. A code consisting of a pixel matrix smaller than the subband A value obtained by dividing the sum of the absolute values of the coefficient values of the 1LH code block by the sum of the absolute values of the coefficient values of the 1HL code block at the position corresponding to the code block is divided into blocks. In step S44, it is determined that the subject is moving when it is larger than the predetermined first threshold value (step S45), and it is determined that the subject is not moving if it is equal to or smaller than the first threshold value. Among the code blocks determined to be moving, a value determined based on the total number of code blocks in which the divided value is greater than a third threshold value that is greater than the first threshold value (step S48), when the moving speed of the subject in the image is higher than the fourth threshold value determined based on the total number of code blocks determined to be moving (S48), Step S51), it is determined that the moving speed of the subject in the image is low when it is equal to or lower than the fourth threshold value, and the data deletion processing means determines that the moving speed of the subject is high by the determining means. For the determined image, data deletion processing is performed in which the data deletion amount of the 1LH subband is smaller than that of the image determined to be low-speed, and the encoding processing means includes data deletion processing means. The encoding processing is executed using data based on the wavelet coefficients that have been subjected to data deletion processing according to the above.
The non-interlaced image processing apparatus according to claim 4 is the non-interlaced image processing apparatus according to claim 3, wherein the determination unit divides the code block determined to be moving. If the value is larger than the third threshold value that is larger than the first threshold value, it is determined that the subject is moving at high speed (step S47), and if the value is equal to or smaller than the third threshold value, the subject is slow. The total number of code blocks determined to be moving at high speed and determined to be moving at high speed (step S48) is a fourth threshold value that is half the total number of code blocks determined to be moving. If it is greater than the threshold value, it is determined that the moving speed of the subject in the image is high (step S51), and if it is equal to or lower than the fourth threshold value, it is determined that the moving speed of the subject in the image is low. It is characterized in.

The non-interlaced image processing apparatus according to claim 5 is the non-interlaced image processing apparatus according to any one of claims 1 to 4, wherein the data deletion processing unit converts the wavelet coefficients into bits. Of the data divided into planes, the data deletion processing is performed by deleting the data corresponding to the lower bit planes in order.
The non-interlaced image processing apparatus according to claim 6 is the non-interlaced image processing apparatus according to any one of claims 1 to 5, wherein the determination unit (step S11) performs the processing. Including provisional arithmetic coding means (steps S61 and S81) for performing provisional coding processing on the wavelet coefficients of the subbands for performing the above processing, and calculating the sum of absolute values of the coefficient values of the subbands for performing the above processing. Instead, the moving speed of the subject in the interlaced image is determined by using the total amount of codes obtained by the provisional arithmetic coding means.

The non-interlaced image processing method according to claim 7, wherein the non-interlaced image is formed by alternately arranging two consecutively scanned interlaced images for each scanned line. A wavelet transform step (step S4) for performing two-dimensional discrete wavelet transform of level 1 or higher on image data, and among the wavelet coefficients obtained by the wavelet transform step , the wavelet coefficients of each subband of level 1 The code block is divided into code blocks each having a pixel matrix smaller than the band, and the absolute value of the coefficient value of the 1HL code block at the position corresponding to the code block is calculated as the absolute value of the coefficient value of the 1LH code block for each code block. The value divided by the sum of the values is S24), it is determined that the subject is moving when it is larger than the predetermined first threshold value (step S25), and it is determined that the subject is not moving when it is equal to or less than the first threshold value. When the value determined based on the sum of the divided values of the code blocks determined to be moving (step S30) is greater than a predetermined second threshold value, It is determined that the moving speed is high (step S31), and if the moving speed of the subject in the image is low when it is equal to or lower than the second threshold, the moving speed of the subject is determined by the determining step. For an image determined to be high speed, a data deletion processing step (step S) for executing a data deletion process in which the data deletion amount of a subband of 1 LH is reduced compared to an image determined to be low speed. ), A coding processing step of performing an encoding process using the data based on wavelet coefficients data deletion process by the data deletion process (step S8), and characterized in that it contains.
The non-interlaced image processing method according to claim 8, wherein the non-interlaced image is formed by alternately arranging two consecutively scanned interlaced images for each scanned line. A wavelet transform process for performing two-dimensional discrete wavelet transform of level 1 or higher on image data (step S4), and among the wavelet coefficients obtained by the wavelet transform process , the wavelet coefficients of each subband of level 1 The code block is divided into code blocks each having a pixel matrix smaller than the band, and the absolute value of the coefficient value of the 1HL code block at the position corresponding to the code block is calculated as the absolute value of the coefficient value of the 1LH code block for each code block. The value divided by the sum of the values is S44), it is determined that the subject is moving if it is larger than the predetermined first threshold value (step S45), and it is determined that the subject is not moving if it is less than the first threshold value. Among the code blocks determined to be moving, a value determined based on the total number of code blocks in which the divided value is greater than a third threshold value that is greater than the first threshold value (step S48), when it is larger than a fourth threshold value determined based on the total number of code blocks determined to be moving, it is determined that the moving speed of the subject in the image is high (step S51). 4 and the determination step of the moving speed of the object in the image when the threshold below is determined to be slow, for a movement speed of the object by determining step is determined to be high-speed image, is slow A data deletion processing step for executing a data deletion process in which the data deletion amount of the 1LH subband is smaller than the determined image (step S6), and data based on the wavelet coefficients subjected to the data deletion processing by the data deletion processing step And an encoding process step (step S8) for executing the encoding process .
The non-interlaced image processing method according to claim 9 is the non-interlaced image processing method according to claim 7 or claim 8, wherein the determination step applies to the wavelet coefficients of subbands to be processed. Code amount obtained by executing temporary arithmetic encoding processing (steps S61 and S81) and performing temporary arithmetic encoding processing instead of the sum of absolute values of the coefficient values of each subband for which the above processing is performed The moving speed of the subject in the interlaced image is determined using the sum of the above.

The non-interlaced image processing apparatus according to any one of claims 1 to 6, wherein a moving speed of a subject in a frame is determined based on a coefficient value of a code block unit of a 1LH subband obtained by two-dimensional discrete wavelet transform. . Thereby, the moving speed of the subject can be determined by a simple calculation process without obtaining the moving amount of the subject between frames. Also, by determining the moving speed of the subject in the frame based on the coefficient value in units of code blocks finer than the subband, for example, only a relatively small subject in a still image may move at high speed. Can be recognized properly.

The non-interlaced image processing method according to any one of claims 7 to 9, wherein a code block unit code amount obtained by arithmetically encoding a 1LH subband coefficient value obtained by two-dimensional discrete wavelet transform. Based on the above, the moving speed of the subject in the frame is determined. Thereby, the moving speed of the subject can be determined by a simple calculation process without obtaining the moving amount of the subject between frames. Also, by determining the moving speed of the subject in the frame based on the coefficient value in units of code blocks finer than the subband, for example, only a relatively small subject in a still image may move at high speed. Can be recognized properly.

(1) Embodiment The interlaced image processing apparatus according to the present invention is an apparatus that executes an encoding process compliant with JPEG2000, and is obtained from a series of a plurality of interlaced images that are continuously captured by a video camera or the like. Image data of one non-interlaced image is processed. Hereinafter, in the text, a non-interlaced image means a single image obtained from the series of a plurality of interlaced images.

  In the image processing apparatus of the present invention, among the wavelet coefficients obtained by performing two-dimensional discrete wavelet transform on the image data of the non-interlaced image, the coefficient value and code amount of the 1HL subband are the coefficient values of the 1LH subband. Focusing on the fact that the code amount increases according to the lateral movement speed of the subject within the frame to be imaged, it shows an almost constant value. The horizontal movement speed (high speed / low speed) is determined in units of code blocks smaller than the subband, for example, a 32 × 32 pixel matrix, and the movement speed of the subject in the frame is determined based on the determination result. Compared to the case of determining the moving speed of a subject in units of subbands, only a relatively small subject in a still image (landscape) is faster by performing a comprehensive speed determination based on the determination result in units of coat blocks. When moving with, it can be recognized correctly.

    DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an image processing apparatus according to the present invention and various modifications of a process for determining a moving speed of a subject within a frame will be described below with reference to the accompanying drawings. FIG. 1 is a diagram illustrating an overall configuration of an interlaced image processing apparatus 10. An interlaced image processing apparatus 10 is composed of a central processing unit (CPU) 1, a ROM 2 that stores an image processing program, a RAM 3 that is used when the image processing program is executed, and a keyboard that is a man-machine interface. 4 and mouse 5, display 6, main recording device, hard disk (HD) 7 for recording image data in the middle of encoding processing of video taken by video camera 8 and encoded data after encoding processing, and video camera 8 is composed.

FIG. 2 is a diagram showing a series of frames taken by the video camera 8. The video camera 8 scans the frame 0 image in the interlace format together with the shooting start time t ₀ , and scans the frame 1 image in the interlace format 1/60 seconds later. Then, a total of n frames of images are scanned in interlace format in units of 1/60 seconds until the end time t _n .

  3A to 3D are diagrams for explaining a phenomenon that occurs when a non-interlaced image is generated from an interlaced image obtained by the video camera 8, and a principle of determination of a moving speed that uses the phenomenon. . As shown in FIG. 3A, in the interlaced format, after scanning a line of one pixel (scan line indicated by a solid line), the line of the pixel immediately below (scan line indicated by a dotted line) is skipped and two pixels below. Are scanned (scan lines indicated by solid lines). After the scan of the interlaced image A is completed, the video camera 8 immediately scans the pixel lines (scan lines indicated by solid lines) that were not scanned last time, as shown in FIG. Thereby, an interlaced image B is photographed. At the time of this photographing, 1/60 seconds have elapsed since the scanning of the line of the pixel immediately below after scanning a certain line. As can be seen by comparing (a) and (b) in FIG. 3, the subject 15 has moved in the right direction (of course, in some cases leftward) during the 1/60 second. For this reason, as shown in FIG. 3C, comb-shaped shifts of several pixels occur at both ends of the non-interlaced image formed by superimposing the interlaced image B on the interlaced image A.

  Note that the non-interlaced image data is alternately arranged for each line obtained by scanning the image data of the interlaced image A and the interlaced image B (in the above example, for each scanning line of one pixel unit). It can be formed by supplementing data).

  The amount L of the comb-shaped shift shown in FIG. 3C increases in proportion to the moving speed of the subject 15 in the interlaced image. As shown in FIG. 3 (d), the coefficient value of the 1LH subband obtained by two-dimensional discrete wavelet transform of the image data of the non-interlaced image having the comb-shaped shift is the horizontal edge component E1. The total increases, that is, in proportion to the moving speed of the subject 15 within the interlaced image. The coefficient value of the 1HL subband increases in proportion to the sum of the vertical edge components E2, but the subject movement follows the empirical rule that most subjects move in the horizontal direction in general shooting. Treated as being almost unchanged by speed. As will be described below, the image processing apparatus 10 determines the moving speed of the subject in the interlaced image in units of code blocks using the above characteristics of the coefficient value of the 1LH subband.

  FIG. 4 is a flowchart of the main routine of the image processing program executed by the CPU 1 of the image processing apparatus 10. First, interlaced image data composed of a series of a plurality of frames (see FIG. 2) is acquired by the video camera 8 (step S1). Specifically, interlaced image data scanned by the video camera 8 in units of 1/60 seconds is recorded in the RAM 3 or the hard disk 7. As a method for acquiring interlaced image data of consecutive frames, in addition to using the video camera 8, for example, a series of interlaced image data for a plurality of frames is recorded on the hard disk 7 in a state before compression encoding. It is also possible to read out the data sequentially.

  Interlaced images for two frames scanned continuously are synthesized (so-called interlaced conversion) to form a non-interlaced image as shown in FIG. 3C (step S2). The interlace conversion process is performed, for example, by alternately outputting image data of two consecutively scanned interlace images for each scanned line in the RAM 3. Alternatively, after the previously read interlaced image data is developed into a non-interlaced image data map provided in the RAM 3, the interface image data to be continuously scanned may be written in the data map.

  The image data of the non-interlaced image obtained by the interlace conversion is converted into Y, Cr, Cb color component data (step S3). In the subsequent processing units, the color data of each component is processed in parallel according to the same procedure, but for the sake of simplicity, only the color data of the Y component will be described below.

  The Y component color data is subjected to level 3 two-dimensional discrete wavelet transform, the wavelet coefficients are divided into code blocks, and the obtained wavelet coefficients are recorded in the RAM 3 or the hard disk 7 (step S4). The wavelet coefficient obtained by the processing is subjected to the scalar quantization processing prescribed in JPEG2000, and the processed data is recorded in the RAM 3 or the hard disk 7 (step S5). The processes from step S3 to step S5 are well-known processes performed in accordance with JPEG2000.

  Next, data reduction processing is performed (step S6). This process is unique to the image processing apparatus 10 and, as will be described in detail later, whether the moving speed of the subject is high based on the wavelet coefficient of the 1LH subband recorded in the RAM 3 or the hard disk 7. For the speed determination process for determining whether the speed is low or not based on the information in units of code blocks, and for the image of the code block determined by the process that the subject in the frame is moving at high speed, the RAM 3 or the hard disk 7 is applied to the data after entropy coding recorded in FIG. 7, and the data reduction process that places importance on the HL component is performed on the image that is determined that the subject is moving at a low speed. Refers to the process of applying.

  After the above data reduction processing, the wavelet coefficients divided into blocks are divided into bit planes, subjected to arithmetic coding processing (so-called coefficient modeling processing) according to a procedure prescribed in JPEG2000, and the processed data is stored in the RAM 3 or the hard disk 7 Record (step S7).

  Using the code data obtained by the above arithmetic coding, the code formation defined in JPEG2000 is executed (step S8), and the obtained code data is recorded on the hard disk 7 (step S9). If all the frame processes have not been completed (NO in step S10), the process returns to step S1 and an encoding process for the next image taken by the video camera 8 is executed. On the other hand, when shooting by the video camera 8 is finished and all encoding processes are finished (YES in step S10), the process is finished.

  As described above, the processes other than steps S1, S2, and S6 (steps S3, S4, S5, S7, and S8) are all executed according to the procedure according to the JPEG2000 standard. Along with the processing in step S10, it may be realized by a hardware circuit. This speeds up the processing. Note that there is already an image processing apparatus that implements all the encoding processing compliant with JPEG2000 with a hardware circuit.

  Furthermore, interlace conversion (step S2) and data reduction processing (step S6) may be realized by a hardware circuit. For example, the interlace conversion process includes the first register that holds the image data of the first scanned interlaced image, the second register that holds the image data of the scanned interlaced image, and the second register. Upon completion of data writing, the first relay switch that outputs data alternately from the first and second registers line by line to the non-interlaced image memory or buffer memory, and data is output from the first and second registers. The third and fourth registers for holding the interlaced image to be continuously scanned, and the data for the non-interlace alternately from the third and fourth registers at the end of the writing of data to the fourth register. Relay that outputs to the image memory or buffer memory By providing a switch it can be implemented as a hardware circuit. This speeds up the processing.

  FIG. 5 is a flowchart of the data reduction process (FIG. 4, step S6). First, based on the coefficient value of each code block of the 1LH subband of the wavelet coefficient obtained by the level 3 two-dimensional discrete wavelet transform (FIG. 4, step S4), the moving speed of the subject in the frame is high / It is determined whether the speed is low (step S11). The processing contents will be described in detail below.

  As a result of the speed determination process in step S11, if it is determined that the speed is high (YES in step S12), the following data reduction process that emphasizes the LH component is performed. This enables further data reduction while maintaining good reproducibility. Specifically, among the data of the wavelet coefficients after the scalar quantization executed in step S5 of FIG. 4, data for 2 bits from the least significant bit (LSB) when 1LH subband data is expanded on the bit plane is used. Data is deleted (step S13). Furthermore, the data of 1 bit is deleted from the least significant bit (LSB) when the data of the 3LH subband is expanded in the bit plane among the data of the wavelet coefficient after the scalar quantization executed in step S5 of FIG. (Step S14).

  On the other hand, when it is determined that the moving speed of the subject in the interlaced image is low (NO in step S12), the following data reduction process is performed with emphasis on the HL component. This enables further data reduction while maintaining good reproducibility. Specifically, among the data of the wavelet coefficients after the scalar quantization executed in step S5 of FIG. 4, the data corresponding to 3 bits from the least significant bit (LSB) when the 1 LH subband data is expanded on the bit plane. Data is deleted (step S15). When the speed is low, 3LH subband data is not deleted from the data of the wavelet coefficients after the scalar quantization executed in step S5 of FIG. After the above processing, the process returns to the main routine of FIG.

  FIG. 6A is a diagram illustrating a state in which the data of the wavelet coefficient after the scalar quantization is divided into subbands and then decomposed into bit planes for each subband, and FIG. FIG. 5 illustrates the contents of processing executed in steps S13 and S14 when it is determined that the speed is high in step S11 of the flowchart shown in FIG. As shown by hatching in FIG. 6B, data of 2 bits is deleted from the least significant bit among the coefficients of the 1 LH subband decomposed into bit planes. The least significant bit data is deleted from the 3LH subband coefficients.

  FIG. 6C illustrates the contents of processing executed in step S15 when it is determined that the speed is low in step S11. As shown by hatching in FIG. 6C, data of 3 bits is deleted from the least significant bit among the coefficients of 1 LH subband decomposed into bit planes. In this case, the 3LH subband coefficient data is not deleted.

FIG. 7 is a flowchart of the speed determination process (FIG. 5, step S11). First, the wavelet coefficients obtained by the level 3 two-dimensional discrete wavelet transform are divided into, for example, code blocks formed of a 32 × 32 pixel matrix (step S20). FIG. 8 is a diagram illustrating code blocks generated in each subband (1LH, 1HL, 1HH, 2LH, 2HL, 2HH, 3LL, 3LH, 3HL, and 3HH) by the above division processing. The 1LH and 1HL subbands are divided into a total of n code blocks. Also, the 2LH and 2HL subbands are divided into a total of m code blocks. Hereinafter be identified by the values of each code block variable CB in a sub-band, for easy understanding, in the figure, for example, mark the CB = 1 code block of subbands 1LH and CB _1. The value of the variable CB increases, for example, one by one according to a so-called raster scanning order within each subband, as indicated by a plurality of arrows in the 1HL subband region.

  Reference is again made to the flowchart of FIG. The value of the variable CB representing the code block is set to 1 as the initial value, the significant code block count value CBC determined to be moving is set to 0, and the total value TR of the rates obtained for the significant code block is set to 0 Set (step S21). The sum sum1LH of the absolute values of the coefficients of the 1LH code block CB is calculated (step S22). The sum sum1HL of the absolute values of the coefficients of the 1HL code block CB is calculated (step S23). Sum1LH / sum1HL = rate is calculated using the obtained coefficient (step S24). The variable rate can be considered to change approximately in proportion to the moving speed of the subject in the frame. This is because, as described above, the value of the 1LH coefficient increases in proportion to the increase in the edge amount in the horizontal direction of the image, that is, the increase in the moving speed of the subject within the frame, whereas the coefficient of 1HL. This value is a value that is proportional to the amount of edge in the vertical direction of the image, and from experience, the subject moves only in the horizontal direction in most cases, and thus takes a relatively stable value.

  If the value of the obtained variable rate is experimentally greater than the threshold th0 that can be determined that the subject in the code block is moving (YES in step S25), the value of the significant code block count value CBC is set. 1 is added (step S26), and the value of the variable rate is accumulated to the value of the rate total value TR (step S27).

  If the value of the variable rate is equal to or less than the threshold th0 (NO in step S25), or if the speed determination process has not been completed for all n code blocks (NO in step S28), the variable CB is set. After adding 1 (step S29), the process returns to step S22 to continue the process in units of code blocks.

  On the other hand, when the above process is completed for all n code blocks, that is, when the variable CB = n, the number of significant code blocks determined to have moved so far and the total value TR of the variables rate are based on the number TR. A speed determination process is executed (steps S30 to S33).

  After determining the variable rate for all code blocks (YES in step S28), the value of the rate total value TR is divided by the number CBC of significant code blocks determined to be moving to obtain the variable speed. (Step S30). The variable speed is the average value of the variable rate of the code block determined to be moving, and when the value is larger than the experimentally obtained threshold value th1 (YES in step S31), It is determined that the subject is moving at high speed (step S32). On the other hand, if the value of the variable speed is equal to or smaller than the threshold th1 (NO in step S31), it is determined that the subject in the frame is moving at a low speed (step S33).

  According to the processing apparatus 10 configured as described above, a still image (landscape) is obtained by performing a comprehensive speed determination based on a determination result in units of coat blocks, compared to a case in which the moving speed of a subject is determined in units of subbands. When only a relatively small subject moves at a high speed, it can be determined that the speed is correct.

(2) Modification 1
In the image processing apparatus 10, as a comprehensive speed determination process based on the determination result in units of coat blocks, an intra-frame is calculated based on the average value of the variable rate indicating the mobility of the code block determined to be moving. The subject within the frame moves at high speed when it is determined that the majority of the code blocks that are determined to be moving are moving at high speed. Modification 1 of the speed determination process for determining that the speed is being performed will be described.

  First, the wavelet coefficients obtained by the level 3 two-dimensional discrete wavelet transform are divided into code blocks each comprising a 32 × 32 pixel matrix (step S40). The value of the variable CB representing the code block is set to 1 as an initial value, the significant code block count value CBC determined to be moving is set to 0, and among the significant code blocks determined to be moving, The variable HCB of the code block determined that the subject is moving at high speed is set to 0 (step S41). The sum sum1LH of the absolute values of the coefficients of the 1LH code block CB is calculated (step S42). The sum sum1HL of the absolute values of the coefficients of the corresponding code block CB of 1HL is calculated (step S43). Sum1LH / sum1HL = rate is calculated (step S44). If the obtained variable rate is larger than the experimentally obtained threshold value th0 (YES in step S45), it is determined that the subject in the code block CB is moving, and 1 is added to the value of the number of significant code blocks CBC. (Step S46). Further, if the variable rate is larger than the threshold value th1 that is experimentally determined (YES in step S47), the code determined that the subject is moving at high speed among the code blocks determined to be moving. 1 is added to the value of the variable HCB representing the number of blocks (step S48).

  The variable rate is equal to or less than the threshold value th0 (NO in step S45), the threshold value is th1 (NO in step S47), or the data reduction processing corresponding to the moving speed has been completed for all n code blocks. If not (NO in step S49), 1 is added to the variable CB (step S50), and then the process returns to step S42 to continue the determination process of the moving speed in units of code blocks.

  On the other hand, when the above processing is completed for all n code blocks, that is, when variable CB = n (YES in step S49), the number of significant code blocks HCB determined to be moving is moving. It is checked whether the number of significant code blocks determined to be greater than the majority (CBC / 2). If so (YES in step S51), it is determined that the subject in the frame is moving at high speed (step S52). If the number is the same or less (NO in step S51), it is determined that the subject in the frame is moving at a low speed (step S53). After this processing, the process returns to the main routine.

(3) Other Modifications As described above, the moving speed of the subject in the frame is proportional to the value of the wavelet coefficient of 1LH, but is obtained by encoding the wavelet coefficient using a 5: 3 lossless filter. Naturally, it is also proportional to the amount of code data of the 1 LH subband. Hereinafter, the case where the two speed determination processes shown in FIGS. 7 and 9 are performed based on the amount of code obtained by the provisional arithmetic encoding process will be briefly described in order.

  FIG. 10 is a diagram illustrating a processing procedure when the speed determination process illustrated in FIG. 7 is performed based on the amount of codes obtained by the provisional arithmetic encoding process. First, the data of the wavelet coefficient after the scalar quantization executed in step S5 of the main routine of FIG. 4 is divided into code blocks composed of a 32 × 32 pixel matrix (step S60). The wavelet coefficient of each code block is divided into bit planes, and provisional arithmetic coding processing is performed on the data of each bit plane (step S61). The value of the variable CB representing the code block is set to 1 as the initial value, the significant code block count value CBC determined to be moving is set to 0, and the total value TR of the rates obtained for the significant code block is set to 0 Set (step S62). The sum sum1LH of the code amounts of the 1LH code block CB is calculated (step S63). The sum sum1HL of the code amount of the 1HL code block CB is calculated (step S64). Sum1LH / sum1HL = rate is calculated using the obtained coefficient (step S65). When the value of the obtained variable rate is experimentally larger than the threshold th2 that can be determined that the subject in the code block is moving (YES in step S66), the value of the significant code block count value CBC is set. 1 is added (step S67), and the value of the variable rate is accumulated to the value of the rate total value TR (step S68).

  If the value of the variable rate is equal to or less than the threshold th2 (NO in step S66), or if the speed determination process has not been completed for all n code blocks (NO in step S69), 1 is set to the variable CB. Are added (step S70), the process returns to step S63, and the process in units of code blocks is continued.

  On the other hand, when the above-described processing is completed for all n code blocks, that is, when variable CB = n (YES in step S69), the number of significant code blocks determined to have moved so far and their rate A speed determination process based on the total value TR is executed (steps S71 to S74).

  After the determination of the variable rate is made for all n code blocks (YES in step S69), the value of the rate total value TR is divided by the number CBC of significant code blocks determined to be moving, and the variable speed is set. Is obtained (step S71). The variable speed is an average value of the variable rate of the code block determined to be moving, and when the value is larger than the experimentally obtained threshold value th3 (YES in step S72), Is determined to be moving at high speed (step S73).

  On the other hand, when the value of the variable speed is equal to or smaller than the threshold th3 (NO in step S72), it is determined that the subject in the frame is moving at a low speed (step S74).

FIG. 11 is a diagram illustrating a processing procedure when the speed determination process illustrated in FIG. 9 is performed based on the amount of code obtained by the provisional arithmetic encoding process. First, the data of the wavelet coefficient after the scalar quantization executed in step S5 of the main routine of FIG. 4 is divided into code blocks composed of a 32 × 32 pixel matrix (step S80). The wavelet coefficients of each code block are divided into bit planes, and arithmetic coding processing is performed on the data of each bit plane (step S81). The value of the variable CB representing the code block is set to 1 as an initial value, the significant code block count value CBC determined to be moving is set to 0, and among the significant code blocks determined to be moving, The number HCB of code blocks determined to be moving at high speed is set to 0 (step S82). The sum sum1LH of the code amount of the 1LH code block CB is calculated (step S83). The sum sum1HL of the code amount of the corresponding code block CB of 1HL is calculated (step S84). Sum1LH / sum1HL = rate is calculated (step S85). If the obtained variable rate is larger than the experimentally obtained threshold value th2 (YES in step S86), it is determined that the subject in the code block CB is moving, and 1 is added to the value of the number of significant code blocks CBC. (Step S87). Further, when the variable rate is larger than the experimentally obtained threshold value th3 (YES in step S88), the code in which it is determined that the subject is moving at high speed among the code blocks determined to be moving. 1 is added to the value of the variable HCB representing the number of blocks (step S99).

  If the value of the variable rate is less than or equal to the threshold th2 (NO in step S86), if the value is less than or equal to the threshold th3 (NO in step S88), or data reduction corresponding to the moving speed for all n code blocks If the process has not been completed (NO in step S90), 1 is added to the variable CB (step S91), and then the process returns to step S83 to continue the determination process of the moving speed in units of code blocks.

  On the other hand, when the above processing is completed for all n code blocks, that is, when variable CB = n (YES in step S90), the number HCB of significant code blocks determined to be moving is moving. It is checked whether the number of significant code blocks determined to be greater than the majority (CBC / 2). If so (YES in step S92), it is determined that the subject in the frame is moving at high speed (step S93). If the number is the same or less (NO in step S92), it is determined that the subject in the frame is moving at a low speed (step S94). After this processing, the process returns to the main routine.

  As described above, the moving speed of the subject in units of code blocks can be easily determined using the two modified examples shown in FIGS. As a result, compared to determining the moving speed of the subject in units of subbands, for example, when only the subject moves in a still image, the still image portion and the moving image portion are processed separately. can do.

1 is an overall configuration diagram of an image processing apparatus. It is a figure which shows a series of frame images image | photographed by a video camera by 1/60 second unit. (A)-(d) is a figure for demonstrating the phenomenon which arises when producing | generating a non-interlaced image from the interlaced image obtained with a video camera, and the principle of determination of the moving speed using the said phenomenon. It is a figure which shows the main routine of the process which an image processing apparatus performs. It is a flowchart of a data reduction process. (A)-(c) is a figure which shows the mode of the data reduction process performed for every code block based on the determination result of the moving speed of a to-be-photographed object. It is a flowchart of a speed determination process. It is a figure for demonstrating the code block formation of a wavelet coefficient. It is a flowchart of the modification of a speed determination process. It is a flowchart of the modification of a speed determination process. It is a flowchart of the modification of a speed determination process.

Explanation of symbols

1 CPU, 2 ROM, 3 RAM, 4 keyboard, 5 mouse, 6 display, 7 hard disk, 8 video camera, 10 image processing device, 15 subject.

Claims (9)

A non-interlaced image processing device (10) formed by alternately arranging two interlaced images scanned in succession for each scanned line, comprising a wavelet transform unit (step S4) and a determination unit (step S11). ), Data deletion processing means (step S6), and encoding processing means (step S8),
The wavelet transform means performs a two-dimensional discrete wavelet transform of level 1 or higher on the data of the non-interlaced image;
The determination means divides the wavelet coefficients of the subbands of level 1 among the wavelet coefficients obtained by the wavelet transform means into code blocks each composed of a pixel matrix smaller than the subband, and each code block has 1 LH. A value obtained by dividing the sum of the absolute values of the coefficient values of the code block by the sum of the absolute values of the coefficient values of the 1HL code block at the position corresponding to the code block (step S24) is a predetermined first threshold value. If it is larger than the first threshold, it is determined that the subject is moving (step S25), and if it is equal to or less than the first threshold value, it is determined that the subject is not moving, and each code block determined to be moving is the above-described code block. The value determined based on the sum of the divided values (step S30) is greater than the predetermined second threshold value. If it is determined that the moving speed of the subject in the image is high (step S31), it is determined that the moving speed of the subject in the image is low if it is equal to or less than the second threshold value. ,
For the image for which the data deletion processing means has determined that the moving speed of the subject is high by the determination means, the data deletion in which the data deletion amount of the 1 LH subband is reduced compared to the image determined to be low speed. To execute the process,
The encoding processing means executes encoding processing using data based on the wavelet coefficients subjected to data deletion processing by the data deletion processing means.
And a non-interlaced image processing apparatus. A value obtained by dividing the sum of the divided values of the code blocks determined to be moving by the determination means by the total number of code blocks determined to be moving (step S30) is obtained in advance. When it is larger than the set second threshold value, it is determined that the moving speed of the subject in the image is high (step S31), and when it is equal to or lower than the second threshold value, the moving speed of the subject in the image is high. It is judged to be slow,
The non-interlaced image processing apparatus according to claim 1 . A non-interlaced image processing device (10) formed by alternately arranging two interlaced images scanned in succession for each scanned line, comprising a wavelet transform unit (step S4) and a determination unit (step S11). ), Data deletion processing means (step S6), and encoding processing means (step S8),
The wavelet transform means performs a two-dimensional discrete wavelet transform of level 1 or higher on the data of the non-interlaced image;
The determination means divides the wavelet coefficients of the subbands of level 1 among the wavelet coefficients obtained by the wavelet transform means into code blocks each composed of a pixel matrix smaller than the subband, and each code block has 1 LH. A value obtained by dividing the sum of the absolute values of the coefficient values of the code block by the sum of the absolute values of the coefficient values of the 1HL code block at the position corresponding to the code block (step S44) is a predetermined first threshold value. If it is larger than the first threshold, it is determined that the subject is moving (step S45), and if it is equal to or less than the first threshold value, it is determined that the subject is not moving, and each code block determined to be moving is the above-described code block. Of which the divided value is based on the total number of code blocks greater than the third threshold value greater than the first threshold value. If the value determined in step S48 is larger than a fourth threshold value determined based on the total number of code blocks determined to be moving, it is determined that the moving speed of the subject in the image is high. (Step S51), it is determined that the moving speed of the subject in the image is low when it is equal to or lower than the fourth threshold value,
For the image for which the data deletion processing means has determined that the moving speed of the subject is high by the determination means, the data deletion in which the data deletion amount of the 1 LH subband is reduced compared to the image determined to be low speed. To execute the process,
The encoding processing means executes encoding processing using data based on the wavelet coefficients subjected to data deletion processing by the data deletion processing means.
And a non-interlaced image processing apparatus. The code is determined to be moving by the determination means, and the subject is moving at a high speed when the divided value is larger than a third threshold value greater than the first threshold value. Determination (step S47), and if it is less than or equal to the third threshold value, it is determined that the subject is moving at low speed, and the total number of code blocks determined to be moving at high speed (step S48) If it is greater than the fourth threshold value, which is half the total number of code blocks determined to be present, it is determined that the moving speed of the subject in the image is high (step S51), and is below the fourth threshold value. In this case, it is determined that the moving speed of the subject in the image is low.
The non-interlaced image processing apparatus according to claim 3. 5. The data deletion processing unit according to claim 1, wherein the data deletion processing unit performs data deletion processing by sequentially deleting data corresponding to a lower bit plane from among data obtained by dividing wavelet coefficients into bit planes. The non-interlaced image processing apparatus according to claim 1. The determination means (step S11) includes provisional arithmetic coding means (steps S61 and S81) for executing provisional coding processing on the wavelet coefficients of the subbands for which the above processing is performed. The moving speed of a subject in an interlaced image is determined using a sum of code amounts obtained by provisional arithmetic coding means instead of a sum of absolute values of subband coefficient values. Item 6. The non-interlaced image processing apparatus according to any one of Items 5. A non-interlaced image processing method in which two consecutively scanned interlaced images are alternately arranged for each scanned line,
A wavelet transform step (step S4) for performing two-dimensional discrete wavelet transform of level 1 or higher on the non-interlaced image data;
Of the wavelet coefficients obtained by the wavelet transform process , the wavelet coefficients of the subbands of level 1 are divided into code blocks each composed of a pixel matrix smaller than the subband, and for each code block, the coefficient value of the code block of 1 LH When the value obtained by dividing the sum of the absolute values of the two by the sum of the absolute values of the coefficient values of the 1HL code block at the position corresponding to the code block is larger than the predetermined first threshold value (step S24). It is determined that the subject is moving (step S25), and if it is equal to or less than the first threshold value, it is determined that the subject is not moving, and the divided value of each code block determined to be moving is described above. If the value determined based on the sum of (step S30) is greater than the predetermined second threshold value, The moving speed of the object is determined to be a high speed (step S31), a determination step of moving speed of the object in the image in the case of the following second threshold value is judged to be low speed,
Data deletion processing for executing data deletion processing for an image for which the moving speed of the subject is determined to be high by the determination step, in which the data deletion amount of the 1LH subband is smaller than that of the image determined to be low speed A process (step S6);
An encoding process step (step S8) for executing an encoding process using data based on the wavelet coefficients subjected to the data deletion process by the data deletion process step;
And a non-interlaced image processing method. A non-interlaced image processing method in which two consecutively scanned interlaced images are alternately arranged for each scanned line,
A wavelet transform step for performing two-dimensional discrete wavelet transform of level 1 or higher on the non-interlaced image data (step S4);
Of the wavelet coefficients obtained by the wavelet transform process , the wavelet coefficients of the subbands of level 1 are divided into code blocks each composed of a pixel matrix smaller than the subband, and for each code block, the coefficient value of the code block of 1 LH When the value obtained by dividing the sum of the absolute values of the two by the sum of the absolute values of the coefficient values of the 1HL code block at the position corresponding to the code block is larger than the predetermined first threshold value (step S44). It is determined that the subject is moving (step S45), and if it is equal to or less than the first threshold value, it is determined that the subject is not moving. Is determined based on the total number of code blocks that are greater than the third threshold and greater than the first threshold. (Step S48), it is determined that the moving speed of the subject in the image is high (step S51) when it is larger than a fourth threshold value determined based on the total number of code blocks determined to be moving. ), A determination step of determining that the moving speed of the subject in the image is low when it is equal to or lower than a fourth threshold value;
For the image moving speed is determined to be high-speed subject by determining step, than the image that is determined to be low speed, data deletion process for performing data deletion processing with a reduced data deletion of subbands 1LH Process (step S6),
An encoding process step for executing an encoding process using data based on the wavelet coefficients subjected to the data deletion process by the data deletion process step (step S8);
And a non-interlaced image processing method. The determination step performs temporary arithmetic coding processing (steps S61 and S81) on the wavelet coefficients of the subbands to be processed, instead of the sum of absolute values of the coefficient values of the subbands to be processed. The non-interlaced image according to claim 7 or 8, wherein the moving speed of the subject in the interlaced image is determined using a total amount of codes obtained by performing the provisional arithmetic coding process. Processing method.

Images