JP2001204038A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent a blur in an image with a motion. SOLUTION: An application area of a color noise reduction circuit 110 is controlled by using a motion vector of a motion compensation circuit 1058 of a moving picture compression circuit 105 provided to the image pickup device for a parameter.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an imaging device using a solid-state imaging device.

[0002]

2. Description of the Related Art Color noise that does not exist in a subject may be mixed in an image signal obtained from a solid-state imaging device such as a CCD. As one method of reducing color noise, Japanese Patent Application Laid-Open No. 11-69226 (hereinafter referred to as Publication 1) discloses a method of obtaining color noise from a solid-state imaging device by utilizing the fact that color noise occurs at random positions. A circuit is described in which a signal is converted into a digital signal, stored in a memory as a plurality of frames, and a color noise component is relatively reduced by superimposing them.

[0003]

However, when the technique described in Publication 1 is applied to a moving image, the image is blurred because the preceding and succeeding moving frames are superimposed and output. In view of Publication 1, it is an object of the present invention to reduce color noise without causing “blur”.

[0004]

When a color noise reduction circuit described in Publication 1 which superimposes images of a plurality of frames is applied to a moving image, a blur occurs in a motion area in the image. . On the other hand, in a moving area in a moving image, it is difficult for human eyes to recognize color noise.

Accordingly, the present invention detects the movement of a subject in a moving image, suppresses blurring without applying a color noise reduction circuit to a moving part, and applies a color noise reduction circuit to a part with little movement. To do. An area in a moving image to which the color noise reduction circuit is not applied is a moving area, and therefore does not visually bother without applying the color noise reduction circuit.

In addition, the motion of a subject in a moving image is detected, and when the motion is large, the color noise reduction circuit is operated, and when the motion is small, the color noise reduction circuit is operated.

The motion of a subject in a moving image is detected by using a motion vector obtained by a motion compensation circuit of an MPEG (Moving Picture Experts Group) compression circuit for compressing the moving image.

Further, as a parameter to be given to the color noise reduction circuit, in addition to the motion vector obtained by the motion compensation circuit, difference information between frames output from the circuit for obtaining correlation between frames in the compression circuit may be added. Good. As difference information between frames, a Y (luminance) signal,
One or more of U, V (color difference) signals are used.

Further, the motion of the subject in the moving image may be detected not by using the motion vector obtained by the motion compensation circuit but by using the motion vector detected by the camera shake correction circuit.

By these means, it is possible to realize an image pickup apparatus provided with a color noise reduction circuit which does not cause blur.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described with reference to FIG. FIG. 1 shows a configuration of a moving image capturing apparatus to which the present invention is applied.

A solid-state image pickup device 102 is formed by a lens 101.
The optical image of the subject formed above is output as an analog electric signal from a solid-state imaging device 102 such as a CCD,
The signal is converted into a digital video signal by the A / D conversion circuit 103. The digital signal is supplied to the luminance / tone adjustment circuit 10.
The adjustment of brightness, white balance, etc. is performed by 4 and input to the compression circuit 105. The compression circuit 105
A memory 106 for storing a plurality of images for each frame,
Difference circuit 10 for calculating a difference between images and outputting a difference image
52, a discrete cosine transform (DCT) circuit 1053 for decomposing the difference image into frequency components, and a quantization (Q) circuit 105 for quantizing the frequency component data decomposed into frequency components
4. An encoding (VLC) circuit 1055 for performing variable length encoding on the quantized quantized data, and an inverse quantization (IQ) circuit 1 for inversely quantizing the quantized data to obtain frequency component data
056, an inverse discrete cosine transform (IDCT) circuit 1057 for obtaining a digital image from frequency component data, and a motion compensation (MC) circuit 1058 for extracting a motion vector from between frames of the digital image and adding motion information to a difference signal. I have. The compressed digital signal output from the compression circuit 105 is recorded on a recording medium such as a magneto-optical disk or a solid-state memory or transmitted to a communication line depending on the application. The color noise reduction circuit 110 uses the memory 106 used by the compression circuit 105 to perform color noise reduction processing based on a plurality of frames stored in the memory 106 for compression processing. Here, the color noise reduction processing and the motion compensation processing using the memory 106 are exclusively performed.

The application area control circuit 107 has a function of controlling an image area on which the color noise reduction circuit 110 operates, using the motion vector information 108 from the motion compensation circuit 1058 in the compression circuit as a parameter. The difference information 10
Reference numeral 9 denotes image difference information between consecutive frames, which will be described in a second embodiment.

FIG. 2 is an example showing the operation of a color noise reduction circuit of the type using a plurality of frames. In each of two consecutive frames 201 and 202 in the moving image, a moving subject 203 and a non-moving subject 20
4. There is color noise 205 due to the solid-state imaging device. Squares 201 and 202 indicate processing units of the image compression processing, and are 16 × 16 pixels in this example. The color noise 205 caused by the solid-state imaging device has a property of being generated at a random position for each frame. Therefore, the frame 201 and the frame 202 are added, and the combined frame 20 is added.
6 is generated. When generating the synthesized frame 206, it is general to perform a division process so that the number of bits of each pixel becomes equal to the frames 201 and 202 before the synthesis. However, the division is not performed and the number of bits of each pixel is increased. There is also a way to do this. In the present embodiment, it is assumed that the division process is performed by the number of combined frames. The value obtained by adding each pixel is divided by the number of frames added. In this embodiment, the divisor is 2 because two frames are added. An image that has been subjected to addition and division processing between two consecutive frames is

[0015]

[Table 1]

An operation result as shown in Table 1 is obtained. Table 1
9 is a table showing changes in the values of each subject and noise in the synthesis processing that is the color noise reduction processing. Although there is no change in the term 2 which is a calculation for an object with no movement, according to the term 3 which is a calculation result for color noise generated at a random position, the signal level is reduced to 、 and the noise is reduced. However, at the same time, according to the term 1 which is an operation for a moving subject, the signal level of the moving subject also becomes １ ／. This action visually causes “movement” of the moving subject, and looks as shown in the result image 206.

FIG. 3 is a diagram for explaining the operation of the application area control circuit of the color noise reduction circuit which is a feature of this embodiment. In each of two consecutive frames 301 and 302 in the moving image, there is a moving subject 303, a non-moving subject 304, and a color noise 305 caused by the solid-state imaging device. Squares 301 and 302 indicate processing units of the image compression processing, which are 16 × 16 pixels. The vector 306 corresponds to the motion compensation circuit 10 in FIG.
The motion vector of the subject detected by 58 is obtained for each 16 × 16 pixel which is a processing unit of the image compression processing, and the image area indicated by the vector is also 16 × 16
Pixel. However, the position indicated by the vector does not necessarily have to match the partition position of a 16 × 16 pixel square, which is the processing unit of the image compression processing. This vector 301
Is information output from the motion compensation circuit 1058 in FIG. The application area control circuit 107 indicates the location of the vector 301 detected by the motion compensation circuit 1058 where the vector 301 exceeds a certain threshold value (for example, the sum of the absolute values of the vertical component and the horizontal component is 2 pixels or more). An area other than the original 16 × 16 pixel area and the 16 × 16 pixel area indicated by the vector 301 is set as an application area 307. The color noise reduction circuit 110 in FIG. 1 performs the addition processing only for the set application area 307 as a color noise reduction processing target area. The operation result is shown in a result image 308. As compared with the result image 210 in FIG. 2, no blur has occurred.

Next, a second embodiment of the present invention will be described. In the second embodiment, the motion vector 108 shown in FIG.
Is added. The application area control circuit 107 uses difference information 109 as a parameter in addition to the motion vector information 108 from the motion compensation circuit 1058. The control method will be described with reference to FIG.

Two consecutive frames 401 in the moving image,
In 402, a moving subject 403 and a non-moving subject 404, and color noise 4 caused by the solid-state imaging device
05 exists. Here, the moving subject 403 is
The subject does not move vertically and horizontally on the screen, but moves in the depth direction. For example, an image of a ball flying to the back of the screen is assumed. Vector 406
Is a motion vector of a subject detected by the motion compensation circuit 1058 in FIG. 1 and is obtained for each 16 × 16 pixel which is a processing unit of the image compression processing in this example, and the image area indicated by the vector is also 16 × 16 Pixel. Here, the position indicated by the vector does not necessarily have to match the partition position of a 16 × 16 pixel square, which is the processing unit of the image compression processing. This vector 401 is shown in FIG.
From the motion compensation circuit 1058 to the application area control circuit 1
07 is output. However, the frame 40
Since there is no subject moving vertically and horizontally between 1,402, both the direction and magnitude of the motion vector are close to zero. Further, the difference information 408 indicates a point where the difference between the luminance signals of the images between the frames 401 and 402 is equal to or larger than a certain threshold value (for example, the sum of the absolute values of the difference detection of each pixel in 16 × 16 pixels is 256). Information.

In the second embodiment, the applicable area control circuit 1
07 is the original 16 × point indicated by the vector 401 at the point where the vector 406 exceeds a certain threshold (for example, the sum of the absolute values of the vertical component and the horizontal component is 2 pixels or more).
16 pixel area and 16 × 16 indicated by vector 401
A region other than the pixel region and a region for each 16 × 16 pixel that does not include the difference information 408 are set as the application region 407. The color noise reduction circuit 110 in FIG.
Performs color noise reduction processing only on the set application area 407. The operation result is shown in a result image 409. According to the second embodiment, the application range of the color noise reduction circuit is determined using the difference information between frames in addition to the motion information, so that more accurate application of the color noise reduction circuit can be realized.

In the first and second embodiments described above,
The application area control circuit 107 changes the application area of the color noise reduction circuit 110 according to the motion vector of each area. In these embodiments, it is needless to say that the application area of the color noise reduction circuit 110 may be 0 when there is movement on the entire screen. When a large motion vector equal to or larger than a predetermined value is detected instead of the application area control circuit 107, the application area is not changed, and the color noise reduction circuit 110 is not applied to the entire screen. An application control circuit that controls the reduction circuit 110 so as not to operate may be provided. When the motion vector is equal to or smaller than the predetermined value, the application control circuit controls the color noise reduction circuit 110 to process the signal of the entire screen. By providing the application control circuit, it is possible to obtain an effect of preventing a screen from being shaken in a scene where there is a large movement and removing color noise in a scene where there is no movement.

Next, a third embodiment will be described. In the first embodiment and the second embodiment, the motion vector of the compression circuit is used to detect the motion. In this embodiment, the motion vector detected by the camera shake correction circuit for correcting the camera shake is used. Use A method of detecting a motion vector in a camera shake correction circuit is described in Japanese Patent Application Laid-Open No. H11-275466.
The description is omitted because it is described in the issue number.

When the motion vector is detected by dividing the screen into a plurality of regions by the camera shake correction circuit, the color noise reduction circuit 110 is controlled as follows. For an area in which the motion vector detected by the camera shake correction circuit exceeds a certain threshold, the applicable area limiting circuit 107 controls so that the color noise reduction circuit 110 is not applied. For a region where the motion vector detected by the camera shake correction circuit does not exceed a certain threshold, the applicable region limiting circuit 107 controls the color noise reduction circuit 110 to apply. Also in the third embodiment, similarly to the first and second embodiments, it is possible to prevent blurring in a moving image and effectively reduce color noise in a region of the moving image where motion is small. .

When the motion vector detected by the camera shake correction circuit is the motion of the entire screen, the color noise reduction circuit 110 is controlled as follows. If the motion vector detected by the camera shake correction circuit exceeds a certain threshold, that is, if the motion of the entire screen is large, the application control circuit controls the color noise reduction circuit 110 so as not to operate on the entire screen. If the motion vector detected by the camera shake correction circuit is below a certain threshold, that is, if the motion of the entire screen is small, the application control circuit
The color noise reduction circuit 110 is controlled to operate. By providing the application control circuit, it is possible to obtain an effect of preventing a screen from being shaken in a scene where there is a large movement and removing color noise in a scene where there is no movement.

As described above, in the first to third embodiments,
Although it has been described that color noise is reduced by overlapping a plurality of frames, color noise may be reduced by overlapping a plurality of fields by performing correlation between a plurality of fields and motion compensation between the fields. . Further, an example in which a motion vector detected by an MPEG compression circuit or a camera shake correction circuit is used has been described. However, the present invention is not limited to this, and a configuration may be adopted in which a motion detection circuit that detects a motion of a subject is specially provided. Although the memory of the compression circuit is used for the color noise reduction calculation, the memory of the color noise reduction circuit may be provided separately instead of the memory of the compression circuit, and compression is not performed. It goes without saying that this can be applied to an imaging device.

[0026]

According to the present invention, blurring in a moving image can be reduced, and color noise can be effectively reduced in a portion having little motion.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an imaging apparatus according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing problems when a conventional color noise reduction process for a still image is applied to a moving image.

FIG. 3 is an operation explanatory diagram for explaining one embodiment of the present invention.

FIG. 4 is an operation explanatory diagram for explaining another embodiment of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 101 ... Lens, 102 ... Solid-state image sensor, 103 ... Analog-digital converter, 104 ... Brightness / tone adjustment circuit, 105 ... Video compression circuit, 1052 ... Difference circuit, 10
53: discrete cosine transform circuit, 1054: quantization circuit,
1055: variable length coding circuit, 1056: inverse quantization circuit, 1057: inverse discrete cosine transform circuit, 1058: motion compensation circuit, 106: memory, 107: application area control circuit, 110: color noise reduction circuit, 201: moving image One frame in the moving image, 202 ... 201, one frame in the moving image, 203: a moving subject, 204: a non-moving subject, 205 ... color noise, 206 ... 201 and 20
2; 301, 1 frame in moving image, 30
2. One frame in a moving image continuous with 301, 303: a moving subject, 304: a non-moving subject, 305 ...
Color noise, 306... Motion vector between 301 and 302, 307...
8: Frame after synthesis, 401: 1 frame in moving image,
402, one frame in a moving image continuous with 401, 403
… A moving subject, 404… a non-moving subject, 40
5 ... color noise, 406 ... motion vector between 401 and 402, 407 ... color noise reduction processing applicable range, 4
08: difference between 401 and 402, 409: frame after synthesis

 ──────────────────────────────────────────────────続 き Continued on the front page F-term (reference) 5C022 AB37 AB55 AC42 5C065 AA01 BB22 BB39 CC01 CC09 CC10 DD02 GG18 GG22 GG27 GG49 GG50 5C066 AA01 AA11 BA20 CA07 CA17 DD07 EC12 EF11 EF12 GA32 GA33 HA02 KA11 KD06 KE11 KE06

Claims (5)

[Claims] An image sensor that photoelectrically converts an optical image; a color noise reduction circuit that reduces color noise by superimposing image signals of a plurality of frames or fields output from the image sensor; Or a motion detection circuit that detects motion between fields, and an application area control circuit that changes an application area of the color noise reduction circuit based on a detection result of the motion detection circuit. apparatus. 2. An image sensor for photoelectrically converting an optical image, a color noise reduction circuit for reducing color noise by superimposing image signals of a plurality of frames or fields output from the image sensor, and An imaging apparatus comprising: a motion detection circuit that detects motion between fields; and an application control circuit that controls the color noise reduction circuit so as not to operate based on a detection result of the motion detection circuit. . 3. A compression circuit for performing a compression process on an image signal output from the image sensor, wherein the detection result of the motion detection circuit is an image signal between the plurality of frames or a field output from the image sensor. The imaging apparatus according to claim 1, wherein the motion vector is a motion vector of a motion compensation circuit that detects a motion vector from the motion vector. 4. A camera shake detection circuit for detecting a camera shake of an image signal output from the image sensor, wherein a detection result of the motion detection circuit is a motion vector detected by the camera shake detection circuit. The imaging device according to claim 1 or 2, wherein 5. The image processing apparatus according to claim 1, further comprising: a difference circuit that calculates a correlation between the plurality of frames or fields output from the imaging device; and a motion compensation circuit that compensates for motion between the plurality of frames or fields. A compression circuit that performs a compression process on an image signal output from the image sensor, wherein the application area control circuit is configured to perform the motion vector detection by the motion compensation circuit and the difference vector detected by the difference circuit. The imaging apparatus according to claim 1, wherein an application area of the color noise reduction circuit is changed.