JP4077706B2 - Interlaced image processing apparatus and processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a moving image processing apparatus, and more particularly to an image processing apparatus that processes a series of interlaced images.
[0002]
[Prior art]
In recent years, JEPPG2000 is known as a compression encoding method suitable for handling high-definition images. There is also a standard called Motion JPEG 2000 that displays a moving image by continuously playing still images encoded in the JPEG 2000 format.
[0003]
[Problems to be solved by the invention]
In a series of interlaced images taken by a video camera or the like, there is a parameter that does not exist in a still image, that is, the amount of movement (change amount) of a subject between frames in comparison with the interlaced image of the previous frame. Several Motion JPEG 2000-format moving image processing apparatuses that detect the moving speed of a subject in a frame using the parameter of the moving amount and adaptively perform compression encoding processing have already been proposed.
[0004]
The motion image processing apparatus in the Motion JPEG 2000 format obtains a difference in subject image data between frames and performs a calculation process for obtaining a moving speed based on the obtained difference data. For this reason, the amount of data to be processed is large, time is required for calculation, and a large amount of memory is required.
[0005]
The present invention determines the moving speed of the subject by using a small amount of data and a simple calculation process without using the difference of the subject image data between the frames, and based on the determination result. An object of the present invention is to provide an image processing apparatus that performs processing.
[0006]
  The interlaced image processing apparatus according to claim 1, an interlace converting unit that forms a non-interlaced image from a series of interlaced images, and a two-dimensional discrete wavelet transform for non-interlaced image data obtained by the interlaced converting unit. A wavelet transform unit for performing, a determination unit for determining a moving speed of a subject in an interlaced image based on a sum of absolute values of coefficient values of 1LH subbands among wavelet coefficients obtained by the wavelet transform unit; And a first encoding processing means for executing the wavelet coefficient encoding processing based on the moving speed of the subject determined by the means.
  An interlaced image processing apparatus according to a second aspect is the interlaced image processing apparatus according to the first aspect, wherein the determination unit adds the absolute values of the count values of the subbands of 1LH ( sum1LH ) Is the sum of the absolute values of 1HL subband counts ( sum1HL ) Divided by ( speed ) Is the first threshold ( Vth1 ) Is larger than the first threshold ( Vth1 ) It is determined that the moving speed of the subject is low at the following times.
  The interlaced image processing apparatus according to claim 3 is the interlaced image processing apparatus according to claim 1, wherein the wavelet transform unit performs two-dimensional discrete wavelet transform of level 2 or higher, The determination means adds the absolute values of the count values of the 1LH subband ( sum1LH ) Is the sum of the absolute values of 2LH subband counts ( sum2LH ) Divided by ( speed ) Is the second threshold (Vth2) Is larger than the second threshold ( Vth2 ) It is determined that the moving speed of the subject is low at the following times.
  The interlaced image processing apparatus according to claim 4 is the interlaced image processing apparatus according to claim 1, wherein the wavelet transform unit performs two-dimensional discrete wavelet transform of level 2 or higher, The determination means adds the absolute values of the count values of the 1LH subband ( sum1LH ) Is the sum of the absolute values of 1HL subband counts ( sum1HL ) Is the sum of absolute values of 2LH subband counts ( sum2LH ) Is the sum of the absolute values of 2HL subband counts ( sum2HL ) Divided by the value divided by ( speed ) Is the third threshold (Vth3) Is larger than the third threshold ( Vth3 ) It is determined that the moving speed of the subject is low at the following times.
[0007]
  The interlaced image processing apparatus according to claim 5 is the interlaced image processing apparatus according to any one of claims 2 to 4, wherein the determination unit counts each subband to be processed. Instead of the sum of absolute values, the moving speed of the subject in the interlaced image is determined using the sum of the code amounts obtained by performing provisional coding processing on the wavelet coefficients of the subbands that perform the above processing. It is what is to be done.
  An interlaced image processing apparatus according to a sixth aspect is the interface image processing apparatus according to any one of the second to fourth aspects, wherein the first encoding processing means determines the determination result by the determination means. And a data reduction processing means for executing data reduction processing based on wavelet coefficients to be encoded, and a second encoding processing means for performing data encoding after the reduction. Features.
[0008]
  The interlaced image processing method according to claim 7 includes an interlace conversion step of forming a non-interlace image from continuous interlace images, and a two-dimensional discrete wavelet conversion on the non-interlace image data obtained by the interlace conversion step. A wavelet transform step to be performed; a determination step of determining a moving speed of a subject in an interlaced image based on a sum of absolute values of coefficient values of 1LH subbands among wavelet coefficients obtained by the wavelet transform step; and And a first encoding process step of performing the wavelet coefficient encoding process based on the moving speed of the subject determined in the determination process.
  The interlaced image processing method according to claim 8 is the interlaced image processing method according to claim 7, wherein the determination step includes a sum of absolute values of count values of 1 LH subbands ( sum1LH ) Is the sum of the absolute values of 1HL subband counts ( sum1HL ) Divided by ( speed ) Is the first threshold ( Vth1 ) Is larger than the first threshold ( Vth1 ) It is determined that the moving speed of the subject is low at the following times.
  The interlaced image processing method according to claim 9 is the interlaced image processing method according to claim 7, wherein the wavelet transform step performs two-dimensional discrete wavelet transform of level 2 or higher, The determination step is the sum of the absolute values of the count values of the 1LH subband ( sum1LH ) Is the sum of the absolute values of 2LH subband counts ( sum2LH ) Divided by ( speed ) Is the second threshold (Vth2) Is larger than the second threshold ( Vth2 ) It is determined that the moving speed of the subject is low at the following times.
  The interlaced image processing method according to claim 10 is the interlaced image processing method according to claim 7, wherein the wavelet transform step performs two-dimensional discrete wavelet transform of level 2 or higher, The determination step is the sum of the absolute values of the count values of the 1LH subband ( sum1LH ) Is the sum of the absolute values of 1HL subband counts ( sum1HL ) Is the sum of absolute values of 2LH subband counts ( sum2LH ) Is the sum of the absolute values of 2HL subband counts ( sum2HL ) Divided by the value divided by ( speed ) Is the third threshold (Vth3) Is larger than the third threshold ( Vth3 ) It is determined that the moving speed of the subject is low at the following times.
[0009]
  The interlaced image processing method according to claim 11 is the interlaced image processing method according to any one of claims 8 to 10, wherein the determination step includes a count value of each subband on which the processing is performed. Instead of the sum of absolute values, the moving speed of the subject in the interlaced image is determined using the sum of the code amounts obtained by executing the provisional encoding process on the wavelet coefficients of the subbands for which the above processing is performed. It is what is to be done.
  An interlaced image processing method according to a twelfth aspect is the interlaced image processing method according to any one of the eighth to tenth aspects, wherein the first encoding processing step is a result of determination by the determination step. And a data reduction processing step for executing data reduction processing based on wavelet coefficients to be encoded, and a second encoding processing step for performing data encoding after reduction. Features.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
  An image processing apparatus according to the present invention processes image data of one non-interlaced image obtained by synthesizing two frames of interlaced images continuously captured by a video camera or the like, andNon-interlaced imageAmong the wavelet coefficients obtained by performing two-dimensional discrete wavelet transformation on the image data, the coefficient value and code amount of the 1LH subband increase in accordance with the lateral movement speed of the subject within the imaged frame. Focusing on the fact that the coefficient value and code amount of the 1HL sub-band show almost constant values, the lateral movement speed (high speed / low speed) of the subject within the frame is determined using the characteristics, A more effective encoding process is executed according to the determination result.
[0011]
DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an image processing apparatus according to the present invention and various modifications of a process for determining a moving speed of a subject within a frame will be described below with reference to the accompanying drawings. FIG. 1 is a diagram illustrating an overall configuration of an interlaced image processing apparatus 10. An interlaced image processing apparatus 10 is composed of a central processing unit (CPU) 1, a ROM 2 that stores an image processing program, a RAM 3 that is used when the image processing program is executed, and a keyboard that is a man-machine interface. 4 and mouse 5, display 6, main recording device, hard disk (HD) 7 for recording image data in the middle of encoding processing of video taken by video camera 8 and encoded data after encoding processing, and video camera 8 is composed.
[0012]
FIG. 2 is a diagram showing a series of frames taken by the video camera 8. The video camera 8 has a shooting start time t₀At the same time, the frame 0 image is scanned in the interlace format, and after 1/60 seconds, the frame 1 image is scanned in the interlace format. And end time t_nIn the meantime, images of a total of n frames are scanned in interlace format in units of 1/60 second.
[0013]
3A to 3D are diagrams for explaining a phenomenon that occurs when a non-interlaced image is generated from an interlaced image obtained by the video camera 8, and a principle of determination of a moving speed that uses the phenomenon. . As shown in FIG. 3A, in the interlaced format, after scanning a line of one pixel (scan line indicated by a solid line), the line of the pixel immediately below (scan line indicated by a dotted line) is skipped and two pixels below. Are scanned (scan lines indicated by solid lines). After the scan of the interlaced image A is completed, the video camera 8 immediately scans the pixel lines (scan lines indicated by solid lines) that were not scanned last time, as shown in FIG. Thereby, an interlaced image B is photographed. At the time of this photographing, 1/60 seconds have elapsed since the scanning of the line of the pixel immediately below after scanning a certain line. As can be seen by comparing (a) and (b) in FIG. 3, the subject 15 has moved in the right direction (of course, in some cases leftward) during the 1/60 second. For this reason, as shown in FIG. 3C, comb-shaped shifts of several pixels occur at both ends of the non-interlaced image formed by superimposing the interlaced image B on the interlaced image A.
[0014]
Note that the non-interlaced image data is alternately arranged for each line obtained by scanning the image data of the interlaced image A and the interlaced image B (in the above example, for each scanning line of one pixel unit). It can be formed by supplementing data).
[0015]
The amount L of the comb-shaped shift shown in FIG. 3C increases in proportion to the moving speed of the subject 15 in the interlaced image. As shown in FIG. 3 (d), the coefficient value of the 1LH subband obtained by two-dimensional discrete wavelet transform of the image data of the non-interlaced image having the comb-shaped shift is the horizontal edge component E1. The total increases, that is, in proportion to the moving speed of the subject 15 within the interlaced image. The coefficient value of the 1HL subband increases in proportion to the sum of the vertical edge components E2, but the subject movement follows the empirical rule that most subjects move in the horizontal direction in general shooting. Treated as being almost unchanged by speed. As will be described below, the image processing apparatus 10 determines the moving speed of the subject in the interlaced image using the above characteristics of the coefficient value of the 1LH subband.
[0016]
FIG. 4 is a flowchart of the main routine of the image processing program executed by the CPU 1 of the image processing apparatus 10. First, interlaced image data composed of a series of a plurality of frames (see FIG. 2) is acquired by the video camera 8 (step S1). Specifically, interlaced image data scanned by the video camera 8 in units of 1/60 seconds is recorded in the RAM 3 or the hard disk 7. As a method for acquiring interlaced image data of consecutive frames, in addition to using the video camera 8, for example, a series of interlaced image data for a plurality of frames is recorded on the hard disk 7 in a state before compression encoding. It is also possible to read out the data sequentially.
[0017]
Interlaced images for two frames scanned continuously are synthesized (so-called interlaced conversion) to form a non-interlaced image as shown in FIG. 3C (step S2). The interlace conversion process is performed, for example, by alternately outputting image data of two consecutively scanned interlace images for each scanned line in the RAM 3. Alternatively, after the previously read interlaced image data is developed into a non-interlaced image data map provided in the RAM 3, the interface image data to be continuously scanned may be written in the data map.
[0018]
The image data of the non-interlaced image obtained by the interlace conversion is converted into Y, Cr, Cb color component data (step S3). In the subsequent processing units, the color data of each component is processed in parallel in the same procedure, but for the sake of simplicity, only the Y component color data will be described below.
[0019]
The Y component color data is subjected to level 3 two-dimensional discrete wavelet transform, and the obtained wavelet coefficients are recorded in the RAM 3 or the hard disk 7 (step S4). The wavelet coefficient obtained by the processing is subjected to the scalar quantization processing prescribed in JPEG2000, and the processed data is recorded in the RAM 3 or the hard disk 7 (step S5). After scalar quantization, JPEG2000 is subjected to entropy encoding processing (so-called coefficient modeling processing) according to a prescribed procedure, and the processed data is recorded in the RAM 3 or the hard disk 7 (step S6). The processes from step S3 to step S6 are well-known processes performed in accordance with JPEG2000.
[0020]
Next, data reduction processing is performed (step S7). This process is unique to the image processing apparatus 10 and, as will be described in detail later, whether the moving speed of the subject is high based on the wavelet coefficient of the 1LH subband recorded in the RAM 3 or the hard disk 7. Alternatively, for the speed determination process for determining whether the speed is low and the image for which it is determined that the subject is moving at high speed by the process, the entropy-encoded data recorded in the RAM 3 or the hard disk 7 is used. The data reduction process that emphasizes the LH component and the image that is determined that the subject is moving at a low speed refers to the process that performs the data reduction process that emphasizes the HL component.
[0021]
An arithmetic coding process defined in JPEG2000 is executed on the data after the data reduction process (step S8), and the obtained code data is recorded on the hard disk 7 (step S9). If all the frame processes have not been completed (NO in step S10), the process returns to step S1 and an encoding process for the next image taken by the video camera 8 is executed. On the other hand, when shooting by the video camera 8 is finished and all encoding processes are finished (YES in step S10), the process is finished.
[0022]
As described above, the processes other than steps S1, S2, and S7 (steps S3 to S6 and S8) are all executed according to the procedure according to the JPEG2000 standard. These processes are the processes of steps S9 and S10. In addition, it may be realized by a hardware circuit. This speeds up the processing. Note that there is already an image processing apparatus that implements all the encoding processing compliant with JPEG2000 with a hardware circuit.
[0023]
Furthermore, interlace conversion (step S2) and data reduction processing (step S7) may be realized by a hardware circuit. For example, the interlace conversion process includes the first register that holds the image data of the first scanned interlaced image, the second register that holds the image data of the scanned interlaced image, and the second register. Upon completion of data writing, the first relay switch that outputs data alternately from the first and second registers line by line to the non-interlaced image memory or buffer memory, and data is output from the first and second registers. The third and fourth registers for holding the interlaced image to be continuously scanned, and the data for the non-interlace alternately from the third and fourth registers at the end of the writing of data to the fourth register. Relay that outputs to the image memory or buffer memory By providing a switch it can be implemented as a hardware circuit. This speeds up the processing.
[0024]
FIG. 5 is a flowchart of the data reduction process (FIG. 4, step S7). First, based on the coefficient value of the 1LH subband of the wavelet coefficient obtained by the level 3 two-dimensional discrete wavelet transform (FIG. 4, step S4), the moving speed of the subject in the frame is either high speed or low speed. Is determined (step S11). The processing contents will be described in detail below.
[0025]
As a result of the speed determination process in step S11, if it is determined that the speed is high (YES in step S12), the following data reduction process that emphasizes the LH component is performed. This enables further data reduction while maintaining good reproducibility. Specifically, of the entropy-encoded data executed in step S6 of FIG. 4, the 2-bit data is deleted from the least significant bit (LSB) when 1LH subband data is expanded on the bit plane. (Step S13). Further, of the entropy-encoded data executed in step S6 in FIG. 4, 1-bit data is deleted from the least significant bit (LSB) when 3LH subband data is expanded on the bit plane (step S6). S14).
[0026]
On the other hand, when it is determined that the moving speed of the subject in the interlaced image is low (NO in step S12), the following data reduction process is performed with emphasis on the HL component. This enables further data reduction while maintaining good reproducibility. Specifically, among the data after entropy coding executed in step S6 of FIG. 4, the data for 3 bits is deleted from the least significant bit (LSB) when 1LH sub-band data is expanded on the bit plane. (Step S15). In the case of the low speed, 3LH subband data in the data after entropy coding executed in step S6 of FIG. 4 is not deleted. After the above processing, the process returns to the main routine of FIG.
[0027]
6A is a diagram illustrating a state where the data after the entropy coding is decomposed into bit planes, and FIG. 6B is determined to be high speed in step S11 of the flowchart illustrated in FIG. In this case, the processing contents executed in steps S13 and S14 are illustrated. As shown by hatching in FIG. 6B, data of 2 bits is deleted from the least significant bit among the coefficients of the 1 LH subband decomposed into bit planes. The least significant bit data is deleted from the 3LH subband coefficients.
[0028]
FIG. 6C illustrates the contents of processing executed in step S15 when it is determined that the speed is low in step S11. As shown by hatching in FIG. 6C, data of 3 bits is deleted from the least significant bit among the coefficients of 1 LH subband decomposed into bit planes. In this case, the 3LH subband coefficient data is not deleted.
[0029]
FIG. 7 is a flowchart of specific processing contents of the speed determination processing (FIG. 5, step S11). First, sum1LH, which is the sum of absolute values of 1LH coefficients, is obtained from the wavelet coefficients obtained by the level 3 two-dimensional discrete wavelet transform executed in step S4 of the main routine shown in FIG. 4 (step S20). Next, sum1HL, which is the sum of absolute values of 1HL coefficients, is obtained (step S21). The value of sum1LH / sum1HL is obtained as a numerical speed representing the moving speed of the subject in the frame (step S22). The speed can be considered to change substantially in proportion to the moving speed of the subject in the frame. This is because, as described above, the value of the 1LH coefficient increases in proportion to the increase in the edge amount in the horizontal direction of the image, that is, the increase in the moving speed of the subject within the frame, whereas the coefficient of 1HL. This value is a value that is proportional to the amount of edge in the vertical direction of the image, and from experience, the subject moves only in the horizontal direction in most cases, and thus takes a relatively stable value.
[0030]
If the coefficient speed obtained in step S22 is larger than the experimentally determined threshold value Vth1 (YES in step S23), it can be determined that the moving speed of the subject within the frame is high ( Step S24). On the other hand, when the coefficient speed is smaller than the threshold value Vth1 (NO in step S23), it can be determined that the moving speed of the subject within the frame is low (step S25). After the above determination, the process returns to the flowchart of FIG.
[0031]
As described above, in the image processing apparatus 10, among the wavelet coefficients obtained by the level 3 two-dimensional discrete wavelet transform performed on the image data to be encoded, in particular, coefficient values and codes of 1 LH subbands. Focusing on the fact that the coefficient value and code amount of the 1HL sub-band show almost constant values, while the amount increases in accordance with the lateral movement speed of the subject within the frame to be imaged, Is used to detect the moving speed of the subject in the frame in the horizontal direction, and more effective encoding processing is executed according to the detected value. As a result, the moving speed of the object can be detected and adaptively encoded by using a small amount of data and a simple calculation process without using the difference in image data between frames. it can.
[0032]
(2) Modification 1
In the image processing apparatus 10, as shown in the flow of FIG. 7, the moving speed of the subject in the frame is determined based on the coefficient values of the 1LH and 1HL subbands. While the numerical value includes information about the edge in the comb-shaped shift portion, the coefficient value of the 2LH subband is a value obtained in units of lines corresponding to two pixels including the comb-shaped shift. It becomes a relatively stable value regardless of the moving speed. Accordingly, as a modified example of the speed determination process, a description will be given below of determining a moving speed of a subject using a coefficient value of a 2LH subband instead of 1HL.
[0033]
FIG. 8 is a flowchart of Modification 1 of the speed determination process (step S11 in FIG. 5). First, sum1LH, which is the sum of absolute values of 1LH subband coefficients, is obtained from the wavelet coefficients obtained by the two-dimensional discrete wavelet transform executed in step S4 of the main routine shown in FIG. 4 (step S30). Next, sum2LH, which is the sum of absolute values of the coefficients of the 2LH subband, is obtained (step S31). The value of sum1LH / sum2LH is obtained as a numerical value speed representing the moving speed of the subject in the frame (step S32). If the coefficient speed is larger than the experimentally determined threshold value Vth2 (YES in step S33), it is determined that the subject in the frame is moving at high speed (step S34). On the other hand, when the coefficient speed is equal to or lower than the threshold value Vth2 (NO in step S33), it is determined that the subject in the frame is moving at a low speed (step S35). After the above determination, the process returns to the flowchart of FIG.
[0034]
(3) Modification 2
Of the wavelet coefficients, the first level coefficients (1LH, 1HL, 1LL) reflect information on the comb-shaped shift portion appearing in the non-interlace image formed by synthesizing the interlace images. On the other hand, the second-level coefficients (2LH, 2HL, 2LL) are values obtained by calculating the above-mentioned comb-shaped deviation in units of lines corresponding to two pixels, and thus are relatively stable regardless of the moving speed of the subject. It is considered to be a value. In the second modification, among the determination formulas obtained based on the above idea, when a subject having many edge portions extending in the horizontal direction is moving at a low speed or at a low speed, it is erroneously determined as moving at a high speed. In the following, a case will be described in which an object that determines the moving speed of an object in a frame is determined by using a determination expression that can most accurately experimentally determine that the object is moving at high speed in the frame.
[0035]
FIG. 9 is a flowchart of Modification 2 of the speed determination process (step S11 in FIG. 5). First, sum1LH, which is the sum of absolute values of 1LH coefficients, is obtained from the wavelet coefficients obtained by the two-dimensional discrete wavelet transform executed in step S4 of the main routine shown in FIG. 4 (step S40). Sum1HL, which is the sum of absolute values of 1HL coefficients, is obtained (step S41). Sum2LH, which is the sum of absolute values of 2LH coefficients, is obtained (step S42). Sum2HL, which is the sum of the absolute values of the 2HL coefficients, is obtained (step S43). The value of (sum1LH / sum1HL) / (sum2LH / sum2HL) is obtained as a numerical speed representing the moving speed of the subject in the frame (step S44). If the coefficient speed is greater than the experimentally determined threshold value Vth3 (YES in step S45), it is determined that the subject in the frame is moving at high speed (step S46). On the other hand, when the coefficient speed is equal to or lower than the threshold value Vth3 (NO in step S45), it is determined that the subject in the frame is moving at a low speed (step S47). After the above determination, the process returns to the flowchart of FIG.
[0036]
(4) Other variations
As described above, the moving speed of the object in the frame is proportional to the value of the 1LH wavelet coefficient, but the code data of the 1LH subband obtained by encoding the wavelet coefficient using a 5: 3 lossless filter. Of course, it is proportional to the amount of. Hereinafter, the case where the three speed determination processes shown in FIGS. 7, 8, and 9 are performed based on the amount of codes obtained by the provisional arithmetic encoding process will be briefly described in order.
[0037]
FIG. 10 is a diagram illustrating a processing procedure when the speed determination process illustrated in FIG. 7 is performed based on the amount of codes obtained by the provisional arithmetic encoding process. First, provisional calculation encoding processing is performed on the data obtained by entropy encoding executed in step S6 of the main routine of FIG. 4 (step S50). Sum1LH which is the sum of the code amount of the data obtained by encoding the coefficients of the 1LH subband among the encoded data is obtained (step S51). Sum1HL, which is the sum of code amounts of data obtained by encoding the coefficients of the 1HL subband, is obtained (step S52). A coefficient speed is obtained by calculating sum1LH / sum1HL (step S53). If the obtained coefficient speed is larger than the experimentally determined threshold value Vth4 (YES in step S54), it is determined that the moving speed of the subject in the frame is high (step S55). On the other hand, when the coefficient speed is equal to or lower than the threshold value Vth4 (NO in step S54), it is determined that the moving speed of the subject in the frame is low (step S56). After the determination process, the process returns to the flowchart of FIG.
[0038]
FIG. 11 is a diagram illustrating a processing procedure when the speed determination process illustrated in FIG. 8 is performed based on the amount of codes obtained by the provisional arithmetic encoding process. First, provisional calculation encoding processing is performed on the data obtained by entropy encoding executed in step S6 of the main routine of FIG. 4 (step S60). Sum1LH, which is the sum of the code amounts of the data obtained by encoding the 1LH subband coefficients in the encoded data, is obtained (step S61). Sum2LH, which is the sum of the code amounts of the data obtained by encoding the coefficients of the 2LH subband, is obtained (step S62). A coefficient speed is obtained by calculating sum1LH / sum2LH (step S63). If the obtained coefficient speed is larger than the experimentally determined threshold value Vth5 (YES in step S64), it is determined that the moving speed of the subject in the frame is high (step S65). On the other hand, when the coefficient speed is equal to or lower than the threshold value Vth5 (NO in step S64), it is determined that the moving speed of the subject in the frame is low (step S66). After the determination process, the process returns to the flowchart of FIG.
[0039]
FIG. 12 is a diagram illustrating a processing procedure when the speed determination process illustrated in FIG. 9 is performed based on the amount of codes obtained by the provisional arithmetic encoding process. First, provisional calculation encoding processing is performed on the data obtained by entropy encoding executed in step S6 of the main routine of FIG. 4 (step S70). Sum1LH, which is the sum of the code amounts of the data obtained by encoding the coefficients of the 1LH subband among the encoded data, is obtained (step S71). Sum1HL, which is the sum of the code amounts of the data obtained by encoding the 1HL subband coefficients, is obtained (step S72). Sum2LH, which is the sum of the code amounts of data obtained by encoding the coefficients of the 2LH subband, is obtained (step S73). Sum2HL, which is the sum of the code amounts of the data obtained by encoding the 2HL subband coefficients, is obtained (step S74). The coefficient speed is obtained by calculating (sum1LH / sum1HL) / (sum2LH / sum2HL) (step S75). If the obtained coefficient speed is larger than the experimentally determined threshold value Vth6 (YES in step S76), it is determined that the moving speed of the subject in the frame is high (step S77). On the other hand, when the coefficient speed is equal to or less than the threshold value Vth6 (NO in step S76), it is determined that the moving speed of the subject in the frame is low (step S78). After the determination process, the process returns to the flowchart of FIG.
[0040]
As described above, the moving speed of the subject in the frame can be easily determined even using the three modified examples shown in FIGS.
[0041]
【The invention's effect】
  In the interlaced image processing apparatus according to any one of claims 1 to 4,Coefficient value of 1LH subband obtained by 2D discrete wavelet transformSum of absolute values ofBased on the above, the moving speed of the subject in the frame is determined. Thereby, the moving speed of the subject can be determined by a simple calculation process without obtaining the moving amount of the subject between frames.
[0042]
  The interlaced image processing apparatus according to claim 5 is the processing apparatus according to any one of claims 2 to 4,Obtained by arithmetically encoding coefficient values of 1 LH subbands obtained by two-dimensional discrete wavelet transformTo the total code amountBased on this, the moving speed of the subject in the frame is determined. Thereby, the moving speed of the subject can be determined by a simple calculation process without obtaining the moving amount of the subject between frames.
  The interlaced image processing apparatus according to claim 6 is the processing apparatus according to any one of claims 2 to 4, and further executes data reduction processing based on a determination result by the determination unit. be able to.
[0043]
  In the interlaced image processing method according to any one of claims 7 to 10,Coefficient value of 1LH subband obtained by 2D discrete wavelet transformSum of absolute values ofBased on the above, the moving speed of the subject in the frame is determined. Thereby, the moving speed of the subject can be determined by a simple calculation process without obtaining the moving amount of the subject between frames.
[0044]
  The interlaced image processing method according to claim 11 is the processing method according to any one of claims 8 to 10,Obtained by arithmetically encoding coefficient values of 1 LH subbands obtained by two-dimensional discrete wavelet transformTo the total code amountBased on this, the moving speed of the subject in the frame is determined. Thereby, the moving speed of the subject can be determined by a simple calculation process without obtaining the moving amount of the subject between frames.
  The interface image processing method according to claim 12 is the processing method according to any one of claims 8 to 10, and further executes a data reduction process based on a determination result of the determination step. be able to.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of an image processing apparatus.
FIG. 2 is a diagram showing a series of frame images taken in 1/60 second units by a video camera.
FIGS. 3A to 3D are diagrams for explaining a phenomenon that occurs when generating a non-interlaced image from an interlaced image obtained by a video camera and a principle of determination of a moving speed that uses the phenomenon. .
FIG. 4 is a diagram illustrating a main routine of processing executed by the image processing apparatus.
FIG. 5 is a flowchart of data reduction processing.
FIGS. 6A to 6C are views showing a state of data reduction processing according to the moving speed of a subject in a frame.
FIG. 7 is a flowchart of speed determination processing.
FIG. 8 is a flowchart of a modified example of the speed determination process.
FIG. 9 is a flowchart of a modified example of the speed determination process.
FIG. 10 is a flowchart of a modified example of the speed determination process.
FIG. 11 is a flowchart of a modified example of the speed determination process.
FIG. 12 is a flowchart of a modified example of the speed determination process.
[Explanation of symbols]
1 CPU, 2 ROM, 3 RAM, 4 keyboard, 5 mouse, 6 display, 7 hard disk, 8 video camera.

Claims (12)

Interlace conversion means for forming a non-interlaced image from a series of interlaced images;
A wavelet transformation means for performing two-dimensional discrete wavelet transform for the data of the non-interlaced image obtained by the interlace conversion means,
Determination means for determining a moving speed of a subject in an interlaced image based on a sum of absolute values of coefficient values of 1LH subbands of wavelet coefficients obtained by the wavelet transform means;
First encoding processing means for executing the wavelet coefficient encoding processing based on the moving speed of the subject determined by the determining means;
A device for processing an interlaced image, comprising: The interlaced image processing apparatus according to claim 1,
The determination means adds the absolute values of the count values of the 1LH subband ( sum1LH ) Is the sum of the absolute values of 1HL subband counts ( sum1HL ) Divided by ( speed ) Is the first threshold ( Vth1 ) Is larger than the first threshold ( Vth1 ) An interlaced image processing apparatus that determines that the moving speed of a subject is low when: The interlaced image processing apparatus according to claim 1,
The wavelet transform means performs level 2 or higher two-dimensional discrete wavelet transform;
The determination means adds the absolute values of the count values of the 1LH subband ( sum1LH ) Is the sum of the absolute values of 2LH subband counts ( sum2LH ) Divided by ( speed ) Is the second threshold (Vth2) Is larger than the second threshold ( Vth2 ) An interlaced image processing apparatus that determines that the moving speed of a subject is low when: The interlaced image processing apparatus according to claim 1,
The wavelet transform means performs level 2 or higher two-dimensional discrete wavelet transform;
The determination means adds the absolute values of the count values of the 1LH subband ( sum1LH ) Is the sum of the absolute values of 1HL subband counts ( sum1HL ) Is the sum of absolute values of 2LH subband counts ( sum2LH ) Is the sum of the absolute values of 2HL subband counts ( sum2HL ) Divided by the value divided by ( speed ) Is the third threshold (Vth3) Is larger than the third threshold ( Vth3 ) An interlaced image processing apparatus that determines that the moving speed of a subject is low when: An interlaced image processing apparatus according to any one of claims 2 to 4,
Instead of the sum of the absolute values of the count values of the subbands for which the above processing is performed, the determination means performs a provisional encoding process on the wavelet coefficients of the subbands for which the above processing is performed. An interlaced image processing apparatus for determining a moving speed of a subject in an interlaced image using a total. An interlaced image processing apparatus according to any one of claims 1 to 5, wherein
The first encoding processing means includes a data reduction processing means for executing data reduction processing based on wavelet coefficients to be encoded based on a determination result by the determination means, and a second code for performing data encoding after the reduction. And an interlaced image processing apparatus. An interlace conversion process for forming a non-interlaced image from continuous interlaced images;
A wavelet transform step for performing a two-dimensional discrete wavelet transform on the non-interlaced image data obtained by the interlace transform step;
A determination step of determining a moving speed of a subject in an interlaced image based on a sum of absolute values of coefficient values of subbands of 1 LH among wavelet coefficients obtained by the wavelet transform step;
A first encoding processing step for performing the wavelet coefficient encoding processing based on the moving speed of the subject determined in the determination step;
And a method for processing an interlaced image. The interlaced image processing method according to claim 7,
The determination step is the sum of absolute values of count values of 1 LH subbands ( sum1LH ) Is the sum of the absolute values of 1HL subband counts ( sum1HL ) Divided by ( speed ) Is the first threshold ( Vth1 ) Is larger than the first threshold ( Vth1 ) An interlaced image processing method for determining that the moving speed of the subject is low at the following times. The interlaced image processing method according to claim 7,
The wavelet transform step performs level 2 or higher two-dimensional discrete wavelet transform,
The determination step is the sum of absolute values of count values of 1 LH subbands ( sum1LH ) Is the sum of the absolute values of 2LH subband counts ( sum2LH ) Divided by ( speed ) Is the second threshold (Vth2) Is larger than the second threshold ( Vth2 ) An interlaced image processing method for determining that the moving speed of the subject is low at the following times. The interlaced image processing method according to claim 7,
The wavelet transform step performs level 2 or higher two-dimensional discrete wavelet transform,
The determination step is the sum of absolute values of count values of 1 LH subbands ( sum1LH ) Is the sum of the absolute values of 1HL subband counts ( sum1HL ) Is the sum of absolute values of 2LH subband counts ( sum2LH ) Is the sum of the absolute values of 2HL subband counts ( sum2HL ) Divided by the value divided by ( speed ) Is the third threshold (Vth3) Is larger than the third threshold ( Vth3 ) An interlaced image processing method for determining that the moving speed of the subject is low at the following times. An interlaced image processing method according to any one of claims 8 to 10,
In the determination step, instead of the sum of absolute values of the count values of the subbands for which the above processing is performed, the code amount obtained by performing a temporary encoding step on the wavelet coefficients of the subbands for which the above processing is performed A method for processing an interlaced image, which uses the sum to determine the moving speed of a subject in the interlaced image. An interlaced image processing method according to any one of claims 8 to 11, comprising:
The first encoding processing step includes a data reduction processing step for executing data reduction processing based on wavelet coefficients to be encoded based on a determination result in the determination step, and a second code for performing data encoding after the reduction. A process for processing an interlaced image.

Images