JP2010074373A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of performing proper control by calculating a moving amount with high accuracy in accordance with a state. <P>SOLUTION: A noise reduction processing part 60 is provided with: a noise elimination amount calculating part 61 for calculating a noise elimination amount on the basis of a difference value between the input image of the current frame and the output image of the preceding frame; a moving amount calculating part 62 for calculating a moving amount on the basis of the difference value between the input image of the current frame and the output image of the preceding frame; and a noise eliminating part for controlling a noise elimination amount in accordance with the moving amount and subtracting the noise elimination amount from the input image of the current frame. The moving amount calculating part 62 calculates a moving amount by giving weight to the difference value and adding them. The weight is set in accordance with the input image of the current frame or the output image of the preceding frame. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that performs noise reduction processing on an image obtained by imaging.

  In recent years, an imaging device such as a digital still camera or a digital video camera that performs imaging using an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) sensor, or a display device such as a liquid crystal display that displays an image. Is widely spread. And various noise reduction techniques for reducing the noise of the image imaged by such an imaging device and the image displayed by a display apparatus are proposed (refer patent document 1).

  As one of the noise reduction techniques, a three-dimensional noise reduction process is used in which a noise component is obtained from a difference value between two temporally different image signals and noise is reduced by removing the noise component from the image signal. Sometimes. In some cases, the amount of motion is obtained from the difference value between the image signals of two frames, and this is used for noise reduction processing. For example, at locations where the amount of motion is small, control is performed to increase the subtraction amount of the noise component, and noise is effectively removed. On the other hand, at locations where the amount of motion is large, control is performed to reduce the subtraction amount of the noise component to suppress the occurrence of tailing or afterimages.

  A motion amount calculation unit implemented in a noise reduction processing apparatus that performs such noise reduction processing will be described with reference to FIG. FIG. 20 is a schematic diagram illustrating a configuration of a motion amount calculation unit implemented in a conventional noise reduction processing device.

  As illustrated in FIG. 20, the motion amount calculation unit 100 includes a difference value calculation unit 101 that calculates a difference value between image signals of the current frame and the previous frame, and a difference value output from the difference value calculation unit 101. An absolute value calculation unit 102 that calculates an absolute value, and a smoothing process (in particular, a filter coefficient that multiplies each of the target pixel and the surrounding pixels by 1) is smoothed by the absolute value of the difference value output from the absolute value calculation unit 102, And a filter unit 103 that calculates a motion amount by dividing a value obtained by adding the respective values after multiplication by the number of pixels.

JP 2000-115586 A

  However, in the motion amount calculation unit shown in FIG. 20, the motion amount may not be calculated accurately. This case will be described with reference to FIG. FIG. 21 is a schematic diagram illustrating an operation example of a motion amount calculation unit implemented in a conventional noise reduction processing device. FIG. 21 shows a case where the amount of motion is calculated based on 35 pixel values including the target pixel (pixels surrounded by a thick line in the figure) and the surrounding pixels. Further, for simplification of description, the pixel value is assumed to be either 100 or 0. Note that the pixel value of the pixel indicating the object is 100, and the pixel value of the pixel indicating the background is 0.

  FIG. 21A illustrates a case where the object moves from the right to the left without changing the pixel value of the target pixel. In the case illustrated in FIG. 21A, the absolute value of the difference value output from the absolute value calculation unit 102 is 10 pixels (the pixels in the second and third columns from the right) are 100, and the other 25 This pixel becomes 0. Therefore, the amount of motion is 100 × 10 / 35≈29.

  On the other hand, FIG. 21B shows a case where the pixel value of the target pixel fluctuates and the object moves from left to right. In the case shown in FIG. 21B, the absolute value of the difference value output from the absolute value calculation unit 102 is 10 pixels (third and fourth columns from the left), as in the case shown in FIG. Eye pixel) is 100, and the other 25 pixels are 0. Therefore, the amount of motion is 100 × 10 / 35≈29.

  In the case illustrated in FIG. 21A, it is preferable that the pixel value of the target pixel does not change and the amount of motion of the target pixel is small (determined to be stationary). On the other hand, in the case shown in FIG. 21B, it is preferable that the pixel value of the pixel of interest fluctuates and the amount of movement of the pixel of interest becomes large (determined that movement has occurred). However, since both motion amounts have the same value, the same control is performed.

  Therefore, particularly in the case of FIG. 21A, there may occur a case where noise is not sufficiently subtracted. In particular, in the case of FIG. 21B, a change in pixel value due to motion is mistaken as noise, and a process of subtracting the mistaken noise may be performed to cause tailing or afterimage.

  In view of the above problems, an object of the present invention is to provide an imaging apparatus that can perform appropriate control by calculating the amount of movement with high accuracy according to the situation.

  In order to achieve the above object, an imaging apparatus of the present invention includes an imaging unit that captures an image to obtain an image, a noise reduction processing unit that obtains an output image by performing noise reduction processing on the image obtained by the imaging unit, The noise reduction processing unit calculates a motion amount based on a difference value between pixel values of corresponding pixels of the first image and the second image captured at different timings by the imaging unit. A motion amount calculating unit for calculating, and a noise removing unit for obtaining an output image by subtracting a removal value from a pixel value of the pixel of the first image, wherein the motion amount calculating unit includes the first image and the first image. A weight is given to a difference value between pixel values of corresponding pixels of two images, and the amount of motion is calculated based on a summed value, and the weight is calculated based on the first image or the second image. Set and before Noise removal unit, and sets the removal value based on the motion amount.

  In the following embodiments, a filter coefficient will be described as an example of the weight.

  Further, in the imaging apparatus having the above-described configuration, the removal value is calculated based on a difference value between pixel values of corresponding pixels of the first image and the second image, and decreases as the amount of motion increases. And the motion amount calculation unit gives the weight to the difference value between the pixel values of the corresponding pixels of the first image and the second image and adds the weight to the weight. The motion amount may be calculated by dividing by the summed value.

  If comprised in this way, a removal value will be based on the difference value of the pixel value of each pixel of a 1st image and a 2nd image. For this reason, when the pixel value of the pixel of the first image fluctuates due to noise or the like, it is possible to suitably reduce the noise by subtracting the removal value from the pixel value. Also, when the subject moves and the amount of motion increases, if the removal value is subtracted from the pixel value of the pixel of the first image, tailing or afterimage occurs. However, by reducing the removal value as the amount of movement increases as in this configuration, it is possible to suppress the occurrence of tailing and afterimages.

  Further, in the imaging apparatus having the above configuration, the motion amount calculation unit may set the weight based on the first image and calculate the motion amount. In addition, the second image may be an output image that has been captured before the first image and has been subjected to the noise reduction processing.

  In the imaging device having the above-described configuration, the second image is an output image that has been captured before the first image and has been subjected to the noise reduction processing. The weight may be set based on the second image, and the amount of motion may be calculated.

  If comprised in this way, it will become possible to set a weight based on the 2nd image in which noise was reduced. For this reason, it is possible to prevent pixels having abnormal pixel values due to noise or the like from being included in an image for which weights are set. Therefore, it is possible to calculate the motion amount with high accuracy.

  In the imaging device having the above configuration, the motion amount calculation unit is close to a predetermined condition from the weight set based on the first image and the weight set based on the second image. One of the weights may be selected, and the amount of motion may be calculated using the weight.

  If comprised in this way, it will become possible to utilize the better result among the weight calculated from the first image and the weight calculated from the second image. Therefore, it is possible to calculate the motion amount with high accuracy. Note that the weight that is closer to the predetermined condition may be the weight that is the sum of the values closer to the predetermined value.

  When the sum of the weights is large, the pixel values of a large number of pixels are reflected in the motion amount. For this reason, it is difficult to determine the magnitude of the movement of the subject in the target pixel. On the other hand, when the sum of the weights is small, the pixel values of a small number of pixels are reflected in the motion amount. For this reason, when noise or the like is included and the pixel values of these pixels become abnormal, the removal value is inappropriate, and the pixel value of the pixel after the noise reduction processing tends to be inappropriate. Therefore, for example, it is preferable that the above-mentioned predetermined condition is that the value obtained by adding up the weights is a substantially intermediate value in a possible range.

  In the imaging device having the above-described configuration, the motion amount calculation unit may calculate a pixel value of each pixel of a pixel group including a target pixel of the first image or the second image and a peripheral pixel of the target pixel. The amount of motion is calculated based on the setting, and among the pixels provided in the pixel group, the weight is set to be larger for a pixel having a smaller difference from the pixel value of the target pixel. It does not matter if it is done.

  With this configuration, it is possible to calculate the amount of motion with a simple configuration.

  In the imaging device having the above-described configuration, the motion amount calculation unit may calculate a pixel value of each pixel of a pixel group including a target pixel of the first image or the second image and a peripheral pixel of the target pixel. A pixel having a small difference between a pixel value of the pixel of interest and a value obtained by smoothing a pixel value in the vicinity of the pixel of interest among the pixels included in the pixel group. The weight may be set so as to increase.

  With this configuration, even when the pixel value of the target pixel becomes inappropriate due to noise or the like, the weight is accurately set. Therefore, it is possible to calculate the motion amount with high accuracy.

  In the imaging device having the above-described configuration, the motion amount calculation unit may calculate a pixel value of each pixel of a pixel group including a target pixel of the first image or the second image and a peripheral pixel of the target pixel. The motion amount is calculated based on the first component obtained based on the position of the target pixel and the positions of the peripheral pixels, the pixel value of the target pixel, and the peripheral A second component obtained based on a difference value from a pixel value of a pixel may be calculated, and a value obtained by multiplying the first component and the second component may be used as the weight.

  If comprised in this way, it will become possible to calculate a motion amount with sufficient precision. In the following embodiments, a domain filter coefficient will be described as an example of the first component, and a range filter coefficient will be described as the second component.

  In the imaging apparatus having the above-described configuration, the motion amount calculation unit may be set so that the motion amount decreases as the value obtained by adding the weights decreases.

  When the sum of the weights is a slight value, there is a high possibility that the pixel value of the pixel affected by the noise is strongly reflected in the motion amount. Therefore, there is a high possibility that the amount of motion becomes large and noise is not sufficiently reduced. However, with the above configuration, it is possible to reduce the amount of motion and effectively reduce noise.

  According to the present invention, the weight used for calculating the amount of motion is set according to the state of the pixel value of the input image. As a result, the amount of motion can be calculated with high accuracy, noise can be effectively reduced, and the occurrence of tailing, afterimages, and the like can be suppressed.

  An embodiment of an imaging apparatus according to the present invention will be described with reference to the drawings. In addition, as an imaging apparatus, an imaging apparatus capable of recording audio, moving images, and still images, such as a digital camera, will be described as an example.

<< Imaging device >>
First, the configuration of the imaging apparatus will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention.

  As shown in FIG. 1, the imaging apparatus 1 forms an image sensor 2 composed of a solid-state imaging device such as a CCD or CMOS sensor that converts an incident optical image into an electrical signal, and an optical image of a subject on the image sensor 2. And a lens unit 3 that adjusts the amount of light and the like. The lens unit 3 and the image sensor 2 constitute an imaging unit, and an image signal is generated by the imaging unit. The lens unit 3 includes various lenses (not shown) such as a zoom lens and a focus lens, and a diaphragm (not shown) that adjusts the amount of light input to the image sensor 2.

  Further, the imaging apparatus 1 converts an image signal, which is an analog signal output from the image sensor 2, into a digital signal and converts an input sound into an electric signal, and an AFE (Analog Front End) 4 that adjusts the gain. In addition to converting the image signal, which is an R (red), G (green), and B (blue) digital signal output from the AFE 4 into a signal using Y (luminance signal) U and V (color difference signal). An image processing unit 6 that performs various image processing on the image signal, an audio processing unit 7 that converts an audio signal that is an analog signal output from the microphone 5 into a digital signal, and an image signal that is output from the image processing unit 6 The image signal output from the image processing unit 6 and the audio signal from the audio processing unit 7 are subjected to compression encoding processing for still images such as JPEG (Joint Photographic Experts Group) compression method. A compression processing unit 8 that performs compression encoding processing for moving images such as a PEG (Moving Picture Experts Group) compression method, and an external memory 10 that records a compression encoded signal compression encoded by the compression processing unit 8; A driver unit 9 that records or reads an image signal in the external memory 10 and a decompression processing unit 11 that decompresses and decodes the compressed and encoded signal read from the external memory 10 in the driver unit 9 are provided. The image processing unit 6 includes a noise reduction processing unit 60 that performs noise reduction processing on the input image signal.

  The imaging device 1 also includes an image output circuit unit 12 that converts the image signal decoded by the expansion processing unit 11 into a signal that can be displayed on a display device (not shown) such as a display, and the expansion processing unit 11 decodes the image signal. And an audio output circuit unit 13 that converts the audio signal into a signal that can be reproduced by a reproduction device (not shown) such as a speaker.

  The imaging apparatus 1 also stores a CPU (Central Processing Unit) 14 that controls the entire operation of the imaging apparatus 1 and a memory 15 that stores each program for performing each process and temporarily stores a signal when the program is executed. A timing generator (TG) that outputs a timing control signal for matching the operation timing of the operation unit 16 to which an instruction from a user such as a button for starting imaging or a button for determining various settings is input. The unit 17 includes a bus line 18 for exchanging signals between the CPU 14 and each unit, and a bus line 19 for exchanging signals between the memory 15 and each unit.

  The external memory 10 may be anything as long as it can record image signals and audio signals. For example, a semiconductor memory such as an SD (Secure Digital) card, an optical disk such as a DVD, a magnetic disk such as a hard disk, or the like can be used as the external memory 10. Further, the external memory 10 may be detachable from the imaging device 1.

  Next, the basic operation of the imaging apparatus 1 will be described with reference to FIG. First, the imaging device 1 acquires an image signal that is an electrical signal by photoelectrically converting light incident from the lens unit 3 in the image sensor 2. Then, the image sensor 2 sequentially outputs image signals to the AFE 4 in a predetermined frame period (for example, 1/60 seconds) in synchronization with the timing control signal input from the TG unit 17. Then, the image signal converted from the analog signal to the digital signal by the AFE 4 is input to the image processing unit 6. In the image processing unit 6, the image signal is converted into a signal using YUV, and various image processes such as gradation correction and contour enhancement are performed. The memory 15 operates as a frame memory, and temporarily holds an image signal when the image processing unit 6 performs processing.

  At this time, based on the image signal input to the image processing unit 6, the lens unit 3 adjusts the position of various lenses to adjust the focus, or adjusts the aperture and adjusts the exposure. It is done. This adjustment of focus and exposure is automatically performed based on a predetermined program so as to be in an optimum state, or manually performed based on a user instruction. The noise reduction processing unit 60 provided in the image processing unit 6 performs noise reduction processing on the input image signal.

  In the case of recording moving images, not only image signals but also audio signals are recorded. An audio signal converted into an electrical signal and output by the microphone 5 is input to the audio processing unit 7 and digitized and subjected to processing such as noise removal. The image signal output from the image processing unit 6 and the audio signal output from the audio processing unit 7 are both input to the compression processing unit 8 and compressed by the compression processing unit 8 using a predetermined compression method. At this time, the image signal and the audio signal are temporally associated with each other, and are configured so that the image and the sound are not shifted during reproduction. The compressed image signal and audio signal are recorded in the external memory 10 via the driver unit 9.

  On the other hand, when only a still image or sound is recorded, the image signal or sound signal is compressed by the compression processing unit 8 by a predetermined compression method and recorded in the external memory 10. Note that the processing performed in the image processing unit 6 may be different depending on whether a moving image is recorded or a still image is recorded.

  The compressed image signal and audio signal recorded in the external memory 10 are read out to the expansion processing unit 11 based on a user instruction. The decompression processing unit 11 decompresses the compressed image signal and audio signal, and outputs the image signal to the image output circuit unit 12 and the audio signal to the audio output circuit unit 13, respectively. Then, the image output circuit unit 12 and the audio output circuit unit 13 convert the signal into a signal that can be displayed or reproduced by a display device or a speaker, and output the signal.

  The display device and the speaker may be integrated with the imaging device 1 or may be separated and connected to a terminal provided in the imaging device 1 using a cable or the like. Absent.

  Further, in a so-called preview mode in which the user confirms an image displayed on a display device or the like without recording an image signal, the image output circuit without compressing the image signal output from the image processing unit 6 It may be output to the unit 12. In addition, when recording an image signal of a moving image, the image signal is output to a display device or the like via the image output circuit unit 12 in parallel with being compressed by the compression processing unit 8 and recorded in the external memory 10. It does not matter.

<< Noise reduction processing section >>
Next, the configuration of the noise reduction processing unit 60 shown in FIG. 1 will be described with reference to the drawings. FIG. 2 is a block diagram illustrating a configuration of a noise reduction processing unit provided in the imaging apparatus according to the embodiment of the present invention. In the following, for the sake of concrete description, an image signal that is input to the noise reduction processing unit 60 and subjected to noise reduction processing is expressed as an image and is also referred to as an “input image”. In addition, an image signal output from the noise reduction processing unit 60 is referred to as an “output image”.

  In addition, the order of input and output images that are sequentially input and output is expressed using frames. In particular, an input image to be subjected to noise reduction processing is set as an input image of the current frame, and an output image output by applying noise reduction processing to the input image of the current frame is set as an output image of the current frame. Further, the input image input to the noise reduction processing unit 60 before the input image of the current frame is set as the input image of the previous frame, and the output image output by applying noise reduction processing to the input image of the previous frame is output as the previous image. The output image of the frame.

  As shown in FIG. 2, the noise reduction processing unit 60 calculates a noise removal amount based on the input image of the current frame and the output image of the previous frame temporarily stored in the frame memory 50. 61, a motion amount calculation unit 62 that calculates a motion amount based on the input image of the current frame and the output image of the previous frame temporarily stored in the frame memory 50, and calculates a feedback coefficient based on the motion amount. A feedback coefficient calculation unit 63; and a noise reduction unit 64 that generates an output image of the current frame by reducing noise of the input image of the current frame based on the noise removal amount and the feedback coefficient.

<Noise removal amount calculation unit>
The noise removal amount calculation unit 61 will be described with reference to the drawings. FIG. 3 is a schematic diagram illustrating a configuration of the noise removal amount calculation unit. As illustrated in FIG. 3, the noise removal amount calculation unit 61 includes a difference value calculation unit 61 a that calculates a difference value between the input image of the current frame and the output image of the previous frame, and a difference value obtained from the difference value calculation unit 61 a. A noise removal amount setting unit 61b for setting a noise removal amount based on the noise removal amount. In addition, the difference value calculated | required by the difference value calculation part 61a becomes a value for every pixel.

  For example, the noise removal amount setting unit 61b sets the noise removal amount by a method as shown in FIG. FIG. 4 is a graph illustrating an example of a method for setting the noise removal amount. As illustrated in FIG. 4, the noise removal amount setting unit 61b sets the noise removal amount n (x) based on the difference value x obtained for each pixel. When the noise removal amount is set as shown in FIG. 4, n (x) is expressed by the following formula (1).

  In n (x) shown in the above equation (1), when the absolute value of the difference value x exceeds the threshold th (th> 0), the noise removal amount n (x) is suppressed. For this reason, when a large variation occurs in the pixel value due to the movement of the subject or the imaging device, it is suppressed that the noise removal amount is mistakenly recognized as a variation due to noise and a large amount of noise removal is set. Therefore, it is possible to suppress the occurrence of tailing and afterimages.

<Feedback coefficient calculation unit>
Next, the feedback coefficient calculation unit 63 will be described with reference to the drawings. FIG. 5 is a graph showing an example of a feedback coefficient setting method. As shown in FIG. 5, the feedback coefficient decreases as the amount of motion increases. For example, if the amount of motion is 0, it is 1, and if the amount of motion exceeds the threshold thM, it is 0. Although the details of the configuration of the motion amount calculation unit 62 and the motion amount calculation method will be described later, generally, the absolute value of the difference value between the pixel value of the input image of the current frame and the pixel value of the output image of the previous frame is The greater the amount, the greater the amount of movement.

<Noise reduction part>
As shown in FIG. 2, the noise reduction unit 64 includes a multiplication unit 64a that multiplies a noise removal amount by a feedback coefficient to calculate a removal value, and a subtraction unit 64b that subtracts the removal value from the input image of the current frame. .

  As described above, the feedback coefficient decreases when the amount of motion is large. For this reason, the removal value calculated by multiplying the noise removal amount by the feedback coefficient is a value obtained by greatly reducing the noise removal amount, and is a small value. When the amount of motion is large, the difference value between the input image of the current frame and the output image of the previous frame is considered to be mainly due to blurring caused by the motion of the subject or the imaging device 1. Therefore, the input value of the current frame is output as it is as the output image of the current frame with the removal value being small. Thereby, it is possible to suppress the occurrence of tailing and afterimages.

  On the other hand, if the amount of motion is small, the feedback coefficient increases. For this reason, the removal value calculated by multiplying the noise removal amount by the feedback coefficient does not reduce the noise removal amount so much and is a large value. When the amount of motion is small, the difference value between the input image of the current frame and the output image of the previous frame is considered to be mainly due to randomly generated noise. For this reason, the noise is suitably reduced from the input image of the current frame with a large removal value.

<Motion amount calculation unit>
Next, each example of the motion amount calculation unit 62 will be described below.

(First embodiment)
A first embodiment of the motion amount calculation unit will be described with reference to the drawings. FIG. 6 is a schematic diagram illustrating the configuration of the first example of the motion amount calculation unit. As illustrated in FIG. 6, the motion amount calculation unit 62a of the present embodiment includes a difference value calculation unit 621 that calculates a difference value between the input image of the current frame and the output image of the previous frame for each pixel, and a difference value calculation unit. An absolute value calculation unit 622 that calculates the absolute value of the difference value output from 621, a filter unit 623 that performs a smoothing process on the absolute value of the difference value output from the absolute value calculation unit 622, and calculates a motion amount; A filter coefficient calculation unit 624a that calculates a filter coefficient of the filter unit 623 based on the input image of the current frame.

  For example, the filter unit 623 multiplies each of the pixel of interest and the surrounding pixels by a predetermined filter coefficient, and divides the value obtained by adding the respective values after the multiplication by a predetermined number (hereinafter referred to as a divisor). It is a filter. The filter unit 623 is assumed to have a variable filter coefficient and a variable divisor. A specific configuration example of the filter unit 623 will be described later.

  Next, the operation of the motion amount calculation unit 62a will be described with reference to the drawings. FIG. 7 is a schematic diagram illustrating an operation example of the first embodiment of the motion amount calculation unit. In FIG. 7, the target pixel is indicated by a thick solid line, and the range of pixels to be multiplied by a filter coefficient having a large weight is indicated by a thick broken line. Further, for simplification of description, the pixel value is assumed to be either 100 or 0. Note that the pixel value of the pixel indicating the object is 100, and the pixel value of the pixel indicating the background is 0.

  As shown in FIG. 6, in the motion amount calculation unit 62a of the present embodiment, first, the difference value calculation unit 621 calculates a difference value between the input image of the current frame and the output image of the previous frame for each pixel. At this time, for example, a difference value is calculated for each of 35 pixels (7 rows in the horizontal direction and 5 columns in the vertical direction) including the target pixel and surrounding pixels centered on the target pixel. Then, the obtained difference value is input to the absolute value calculation unit 622, and the absolute value is calculated.

  On the other hand, the filter coefficient calculation unit 624a calculates a filter coefficient based on the input image of the current frame. Specifically, for example, first, a pixel having a pixel value substantially equal to the pixel value of the target pixel (for example, a pixel value whose difference from the pixel value of the target pixel is within a predetermined range) is detected. Then, when the filter unit 623 calculates the amount of motion, the filter coefficient is set so that the weight of the pixel value of the detected pixel becomes large.

  In the following, an example will be described in which the filter coefficient to be multiplied by the pixels detected as described above is set to 1, and the filter coefficient to be multiplied to other pixels is set to 0. Note that the divisor for dividing the value obtained by adding the respective values after multiplication is a value obtained by adding the set filter coefficients (that is, the number of pixels multiplied by filter coefficient = 1 in this example). .

  In the example shown in FIG. 7A, the pixel value of the target pixel of the input image of the current frame is 100. Further, as indicated by a thick broken line in the figure, the number of pixels having a pixel value of 100 is 20 for four columns from the left in the horizontal direction. In this embodiment, each of the absolute values of the difference values of these 20 pixels is multiplied by the filter coefficient = 1. Then, the value obtained by adding the respective values after multiplication is divided by a divisor = 20. In the case shown in FIG. 7A, the absolute value of the difference value of all 20 pixels is zero. Therefore, (0 × 20) / 20 = 0 is the amount of motion.

  In the example shown in FIG. 7B, the pixel value of the target pixel of the input image of the current frame is 100. Further, as indicated by a thick broken line in the figure, the number of pixels having a pixel value of 100 is 20 for four columns from the left in the horizontal direction. In this embodiment, each of the absolute values of the difference values of these 20 pixels is multiplied by the filter coefficient = 1. Then, the value obtained by adding the respective values after multiplication is divided by a divisor = 20. In the case shown in FIG. 7B, the absolute value of the difference value of 10 pixels is 0, and the absolute value of the difference value of 10 pixels is 100. Therefore, the amount of movement is (0 × 10 + 100 × 10) / 20 = 50.

  According to the motion amount calculation unit 62a of the present embodiment, since the motion amount is calculated by increasing the weight of a pixel whose pixel value is substantially equal to the target pixel of the current frame, the motion amount corresponding to the change in the pixel value of the target pixel is calculated. It becomes possible to calculate more accurately.

  Specifically, the cases of FIGS. 7A and 7B will be described. In the case shown in FIG. 7A, it is preferable that the object is moving, but it is determined that the movement is small because the pixel value of the target pixel has not changed. In the case of the present embodiment, the motion amount output from the motion amount calculation unit 62a is 0, and it is determined that the motion is small (still). Therefore, the removal value increases as the feedback coefficient input to the noise reduction unit 64 increases, and noise can be effectively suppressed.

  On the other hand, in the case shown in FIG. 7B, it is preferable to determine that the movement is large because the object has movement and the pixel value of the target pixel fluctuates. In the case of the present embodiment, the motion amount output from the motion amount calculation unit 62a is 50, and it is determined that the motion is large. Accordingly, the feedback value input to the noise reduction unit 64 becomes small, so that the removal value becomes small, and the occurrence of tailing and afterimage can be suppressed.

(Second embodiment)
A second embodiment of the motion amount calculation unit will be described with reference to the drawings. FIG. 8 is a schematic diagram showing the configuration of the second embodiment of the motion amount calculation unit, and corresponds to FIG. 6 shown for the first embodiment. In addition, the same code | symbol is attached | subjected to the part similar to 1st Example, and detailed description is abbreviate | omitted.

  As shown in FIG. 8, the motion amount calculation unit 62b of the present embodiment includes a difference value calculation unit 621, an absolute value calculation unit 622, a filter unit 623, and a filter of the filter unit 623 based on the output image of the previous frame. A filter coefficient calculation unit 624b that calculates a coefficient.

  Next, the operation of the motion amount calculation unit 62b will be described with reference to the drawings. FIG. 9 is a schematic diagram showing an operation example of the second embodiment of the motion amount calculation unit, and corresponds to FIG. 7 shown for the first embodiment. As in FIG. 7, in FIG. 9, the target pixel is indicated by a thick solid line, and the range of pixels to be multiplied by a filter coefficient having a large weight is indicated by a thick broken line. Further, for simplification of description, the pixel value is assumed to be either 100 or 0. Note that the pixel value of the pixel indicating the object is 100, and the pixel value of the pixel indicating the background is 0.

  As shown in FIG. 8, in the motion amount calculation unit 62b of the present embodiment, first, the difference value calculation unit 621 calculates a difference value between the input image of the current frame and the output image of the previous frame for each pixel. For example, the difference value is calculated for each of the 35 pixels as in the first embodiment. Then, the obtained difference value is input to the absolute value calculation unit 622, and the absolute value is calculated.

  On the other hand, the filter coefficient calculation unit 624b calculates a filter coefficient based on the output image of the previous frame. Specifically, for example, first, a pixel having a pixel value substantially equal to the pixel value of the target pixel (for example, a pixel value whose difference from the pixel value of the target pixel is within a predetermined range) is detected. Then, when the filter unit 623 calculates the amount of motion, the filter coefficient is set so that the weight of the pixel value of the detected pixel becomes large. In the following description, similarly to the first embodiment, the filter coefficient to be multiplied by the detected pixel is set to 1, and the filter coefficient to be multiplied to other pixels is set to 0. In addition, the divisor for dividing the value obtained by adding the respective values after multiplication is a value obtained by adding the set filter coefficients (that is, the number of pixels multiplied by filter coefficient = 1 in this example). .

  In the example shown in FIG. 9A, the pixel value of the target pixel of the output image of the previous frame is 100. Further, as indicated by the bold broken line in the figure, the number of pixels having a pixel value of 100 is 30 for six columns from the left in the horizontal direction. In this embodiment, each of the absolute values of the difference values of these 30 pixels is multiplied by the filter coefficient = 1. Then, the value obtained by adding the respective values after multiplication is divided by a divisor = 30. In the case shown in FIG. 9A, the absolute value of the difference value of 20 pixels is 0, and the absolute value of the difference value of 10 pixels is 100. Therefore, (0 × 20 + 100 × 10) / 30≈33 is the amount of motion.

  In the example shown in FIG. 9B, the pixel value of the target pixel of the input image of the current frame is 100. Further, as indicated by a thick broken line in the figure, the number of pixels having a pixel value of 100 is 25 for five columns from the right in the horizontal direction. In this embodiment, each of the absolute values of the difference values of these 25 pixels is multiplied by the filter coefficient = 1. Then, the value obtained by adding the respective values after multiplication is divided by a divisor = 25. In the case shown in FIG. 9B, the absolute value of the difference value of 15 pixels is 0, and the absolute value of the difference value of 10 pixels is 100. Therefore, the amount of movement is (0 × 15 + 100 × 10) / 25 = 40.

  The motion amount calculation unit 62b of the present embodiment calculates the motion amount by increasing the weight of a pixel whose pixel value is substantially equal to the target pixel of the output image of the previous frame. Therefore, similarly to the motion amount calculation unit 62a of the first embodiment, it is possible to calculate the motion amount corresponding to the fluctuation of the pixel value of the target pixel with higher accuracy.

  Specifically, the cases of FIGS. 9A and 9B will be described. In the case illustrated in FIG. 9A, it is preferable that the object is in motion but it is determined that the motion is small because the pixel value of the target pixel has not changed. In the case of the present embodiment, the motion amount output from the motion amount calculation unit 62b is 33, and it is determined that the motion is relatively small. Therefore, the removal value increases as the feedback coefficient input to the noise reduction unit 64 increases, and noise can be effectively suppressed.

  On the other hand, in the case shown in FIG. 9B, it is preferable to determine that the movement is large because the object has movement and the pixel value of the target pixel fluctuates. In the case of the present embodiment, the motion amount output from the motion amount calculation unit 62b is 40, and it is determined that the motion is relatively large. Accordingly, the feedback value input to the noise reduction unit 64 becomes small, so that the removal value becomes small, and the occurrence of tailing and afterimage can be suppressed.

  In addition, the motion amount calculation unit 62b of the present embodiment sets the filter coefficient with reference to the target pixel of the output image of the previous frame that has already been subjected to the noise reduction process. Therefore, it is possible to suppress the influence of noise when calculating the filter coefficient. Specifically, it is possible to prevent the filter coefficient from being accurately calculated because the pixel values of the target pixel and other pixels become abnormal values due to the influence of noise. Therefore, it is possible to calculate the motion amount with higher accuracy.

  Further, with the configuration of the present embodiment, for example, noise can be effectively reduced even in a flat portion where pixel values are uniform. This will be described with reference to the drawings. FIG. 10 is a schematic diagram showing an operation example of the second embodiment of the motion amount calculation unit, which is the same as FIG. FIGS. 10A and 10B show the case where the pixel value fluctuates due to noise. For simplification of description, the pixel values of pixels including noise are all 500.

  FIG. 10A shows a case in which noise is included only in the target pixel and its vicinity (one pixel below the target pixel) in the output image of the previous frame. In this case, the absolute value of the difference value between the two pixels is multiplied by the filter coefficient = 1. Then, the value obtained by adding the respective values after multiplication is divided by the divisor = 2. In the case shown in FIG. 10A, the input image of the current frame is not affected by noise, and all of the pixel values of the 35 pixels for which the motion amount is calculated are zero. Therefore, the absolute value of the difference value between the two pixels is 500 for each. Therefore, (500 × 2) / 2 = 500 is the amount of motion.

  As shown in FIG. 10A, even when no movement occurs, the amount of movement increases due to the influence of noise or the like. In this case, the feedback coefficient is reduced, and feedback of an inappropriate noise removal amount set based on the difference value between the output image of the previous frame including noise and the input image of the current frame is suppressed (the removal value is Smaller). Therefore, the pixel value of the pixel of the output image of the current frame is suppressed from becoming inappropriate.

  FIG. 10B shows a case where noise is included only in the vicinity of the target pixel (two pixels to the left of the target pixel) and no noise is included in the target pixel in the output image of the previous frame. Yes. In this case, the absolute value of the difference value of the 33 pixels excluding the two pixels is multiplied by the filter coefficient = 1. Then, the value obtained by adding the respective values after multiplication is divided by a divisor = 33. In the case shown in FIG. 10B, noise is included only in the target pixel of the input image of the current frame. Therefore, the absolute value of the difference value of the target pixel is 500, and the absolute value of the difference value of the other 32 pixels is 0. Therefore, the amount of motion is (500 × 1 + 0 × 32) / 33≈15.

  As shown in FIG. 10B, when noise is included in pixels other than the target pixel in the output image of the previous frame, the amount of motion becomes small. As a result, the feedback coefficient increases, and the feedback of the noise removal amount increases (the removal value increases). For this reason, even if noise is included in the target pixel of the current frame, the removal value becomes large, so that noise can be effectively reduced.

(Third embodiment)
A third embodiment of the motion amount calculation unit will be described with reference to the drawings. FIG. 11 is a schematic diagram showing the configuration of the third embodiment of the motion amount calculation unit, and corresponds to FIG. 6 shown for the first embodiment. In addition, the same code | symbol is attached | subjected to the part similar to 1st Example, and detailed description is abbreviate | omitted.

  As shown in FIG. 11, the motion amount calculation unit 62c of the present embodiment includes a difference value calculation unit 621, an absolute value calculation unit 622, a filter unit 623c, and a filter of the filter unit 623c based on the input image of the current frame. A filter coefficient calculation unit 624c1 that calculates a coefficient, and a filter coefficient calculation unit 624c2 that calculates the filter coefficient of the filter unit 623c based on the output image of the previous frame. The filter coefficient calculation unit 624c1 is similar to the filter coefficient calculation unit 624a of the motion amount calculation unit 62a of the first embodiment illustrated in FIG. The filter coefficient calculation unit 624c2 is the same as the filter coefficient calculation unit 624b of the motion amount calculation unit 62b of the second embodiment shown in FIG.

  The filter unit 623c selects one of the filter coefficient output from the filter coefficient calculation unit 624c1 and the filter coefficient output from the filter coefficient calculation unit 624c2. Then, smoothing processing is performed on the absolute value of the difference value output from the absolute value calculation unit 622 using the selected filter coefficient, and the amount of motion is calculated.

  Next, the operation of the motion amount calculation unit 62c will be described with reference to the drawings. FIG. 12 is a schematic diagram showing an operation example of the third embodiment of the motion amount calculation unit, and corresponds to FIG. 7 shown for the first embodiment. As in FIG. 7, in FIG. 12, the target pixel is indicated by a thick solid line, and the range of pixels to be multiplied by a filter coefficient having a large weight is indicated by a thick broken line. Further, for simplification of description, the pixel value is assumed to be either 100 or 0. Note that the pixel value of the pixel indicating the object is 100, and the pixel value of the pixel indicating the background is 0.

  As shown in FIG. 11, in the motion amount calculation unit 62 c of this embodiment, first, the difference value calculation unit 621 calculates a difference value between the input image of the current frame and the output image of the previous frame for each pixel. For example, the difference value is calculated for each of the 35 pixels as in the first embodiment. Then, the obtained difference value is input to the absolute value calculation unit 622, and the absolute value is calculated.

  The filter coefficient calculation unit 624c1 calculates a filter coefficient based on the input image of the current frame. Specifically, first, a pixel having a pixel value substantially equal to the pixel value of the target pixel (for example, a pixel value whose difference from the pixel value of the target pixel is within a predetermined range) is detected. Then, when the filter unit 623c calculates the motion amount, the filter coefficient is set so that the weight of the pixel value of the detected pixel becomes large. In the following description, similarly to the first embodiment, the filter coefficient to be multiplied by the detected pixel is set to 1, and the filter coefficient to be multiplied to other pixels is set to 0. In addition, the divisor for dividing the value obtained by adding the respective values after multiplication is a value obtained by adding the set filter coefficients (that is, the number of pixels multiplied by filter coefficient = 1 in this example). . The filter coefficient calculation unit 624c1 performs the same operation as the filter coefficient calculation unit 624a of the first embodiment.

  In addition, the filter coefficient calculation unit 624c2 calculates a filter coefficient based on the output image of the previous frame. The calculation method itself is the same as that of the filter coefficient calculation unit 624c1. That is, a pixel having a pixel value substantially equal to the pixel value of the target pixel is detected, and a filter coefficient is calculated so that the weight of the detected pixel becomes large when the filter unit 623c calculates a motion amount. Further, the divisor that divides the value obtained by adding the respective values after multiplication is a value obtained by adding the set filter coefficients (that is, in this example, the number of pixels multiplied by the filter coefficient = 1). . The filter coefficient calculation unit 624c2 performs the same operation as the filter coefficient calculation unit 624b of the second embodiment.

  The filter unit 623c selects one of the filter coefficient input from the filter coefficient calculation unit 624c1 and the filter coefficient input from the filter coefficient calculation unit 624c2. Specifically, a filter coefficient whose divisor (the number of pixels multiplied by filter coefficient = 1) is closer to a predetermined value j is selected. The predetermined value j is preferably about half the number of absolute values of difference values input to the filter unit 623c (ie, half of 35≈18).

  The case where the predetermined value j is 18 will be described below. In the case illustrated in FIG. 12A, the number of pixels multiplied by the filter coefficient = 1 in the filter coefficient output from the filter coefficient calculation unit 624c1 is 20 (see FIG. 7A). On the other hand, in the filter coefficient output from the filter coefficient calculation unit 624c2, the number of pixels multiplied by the filter coefficient = 1 is 30 (see FIG. 9A). Therefore, in the case shown in FIG. 12A, the filter coefficient output from the filter coefficient calculation unit 624c1 is selected.

  In the case illustrated in FIG. 12B, the number of pixels multiplied by the filter coefficient = 1 in the filter coefficient output from the filter coefficient calculation unit 624c2 is 20 (see FIG. 7B). On the other hand, in the filter coefficient output from the filter coefficient calculation unit 624c2, the number of pixels multiplied by the filter coefficient = 1 is 25 (see FIG. 9B). Therefore, in the case shown in FIG. 12B, the filter coefficient output from the filter coefficient calculation unit 624c1 is selected.

  As described above, the filter coefficient is selected by the filter unit 623c, and the amount of motion is calculated. Specifically, for example, it is calculated as shown in FIG. In the example shown in FIG. 12A, the pixel value of the target pixel of the input image of the current frame is 100, and there are 20 pixels corresponding to four columns from the left in the horizontal direction. Each of the absolute values of the difference values of these 20 pixels is multiplied by a filter coefficient = 1, and a value obtained by adding the values after multiplication is divided by a divisor = 20. Then, (0 × 20) / 20 = 0 is the amount of motion.

  In the example shown in FIG. 12B, the pixel value of the target pixel of the input image of the current frame is 100, and the number of pixels having the pixel value of 100 is 20 for four columns from the left in the horizontal direction. Each of the absolute values of the difference values of these 20 pixels is multiplied by the filter coefficient = 1, and the value obtained by adding the values after multiplication is divided by the divisor = 20. The amount of motion is (0 × 10 + 100 × 10) / 20 = 50.

  The motion amount calculation unit 62c of the present embodiment calculates the motion amount by increasing the weight of a pixel whose pixel value is substantially equal to the target pixel of the input image of the current frame or the output image of the previous frame. Therefore, similarly to the motion amount calculation unit 62a of the first embodiment, it is possible to calculate the motion amount corresponding to the variation of the target pixel with higher accuracy. Therefore, it is possible to effectively reduce noise and to suppress the occurrence of tailing and afterimages.

  Furthermore, the number of pixels for increasing the weight of the filter coefficient can be brought close to an appropriate value (predetermined value j). For example, when the number of pixels multiplied by the filter coefficient = 1 is increased, the amount of motion affected by the pixel values of many pixels is calculated. For this reason, it is difficult to determine the magnitude of the motion in the target pixel. On the other hand, if the number of pixels to be multiplied by the filter coefficient = 1 is reduced, the fluctuation of the pixel value due to noise can be strongly reflected in the amount of motion, so the possibility that the amount of motion becomes an inappropriate value increases. Therefore, by making the filter coefficient selectable as in the motion amount calculation unit 62c of the present embodiment, it is possible to calculate a more accurate motion amount using the optimum filter coefficient.

  In addition, with respect to the motion amount calculation units 62a to 62c of the first to third embodiments, a filter coefficient that is multiplied by a pixel having a pixel value substantially equal to the pixel value of the target pixel is 1, and a filter coefficient that is multiplied by other pixels However, the value is not limited to this value as long as the weight of a pixel having a pixel value substantially equal to the pixel value of the target pixel is large. For example, a filter coefficient that is multiplied by a pixel having a pixel value substantially equal to the pixel value of the target pixel may be v, and a filter coefficient that is multiplied by another pixel may be m (provided that v> m).

  The filter coefficient values that can be selected may be two or more. In particular, a filter coefficient that is multiplied by a pixel having a large difference from the pixel value of the target pixel may be set to a smaller value.

Further, for example, the movement amount M may be calculated as shown in the following formula (2). The following formula (2) is a general formula, and can be applied to the above-described case where the filter coefficient is a binary value of 0 or 1. In the following equation (2), f _k is a filter coefficient for multiplying the pixel value of the k th pixel, p _ck is the pixel value of the k th pixel in the input image of the current frame, and p _pk is the output image of the previous frame. Is the pixel value of the k-th pixel. Further, the case where the motion amount M is calculated based on the pixel values of N + 1 (35 in the above example) pixels from 0 to Nth is shown.

  When the input image input to the noise reduction processing unit 60 has three types of pixel values of R, G, and B, noise reduction processing is performed for each type (that is, for each type). It does not matter. Further, noise reduction processing may be performed on RAW data (data before color synchronization and data having a pixel value of one color per pixel).

  Similarly, when the input image has three types of pixel values Y, U, and V, similarly, noise reduction processing may be performed for each type. Furthermore, the same processing may be performed when the input image includes pixel values of types other than RGB and YUV.

  The motion amount calculation units 62a to 62c may calculate the motion amount based on the luminance of the input image (for example, the pixel value Y or a value corresponding to Y calculated from the RGB pixel values). Absent. The amount of motion may be calculated based on each pixel value to be processed (RGB or YUV).

  In addition, although the image input to the noise reduction processing unit 60 and processed is the input image of the current frame and the output image of the previous frame, other images may be used as long as the imaging timing is different. . For example, the input image of the current frame and the input image of the previous frame may be used, and the images may not be obtained by continuously capturing images.

<< Modification >>
<Configuration example of filter unit>
Next, a configuration example of the filter units 623 and 623c will be described with reference to the drawings. FIG. 13 is a block diagram illustrating a configuration example of the filter unit. Hereinafter, the absolute value of the difference value between the pixel of the input image of the current frame and the pixel of the output image of the previous frame that is input to the filter unit is referred to as a difference value absolute value.

  FIG. 13 may show a part of the filter unit. In particular, FIG. 13 shows a configuration for smoothing the difference value absolute value of pixels for one row aligned in the horizontal direction. Therefore, in the case where the smoothing process is performed on the difference value absolute value of the pixels aligned over a plurality of rows, FIG. 13 shows a part of the filter unit. In the following, the filter unit smoothes the absolute value of the difference value of 35 pixels (5 rows in the vertical direction and 7 columns in the horizontal direction) including the target pixel and surrounding pixels centered on the target pixel. The case where it is the structure which performs a process is demonstrated.

Filter unit (in part) a delay unit d ₁ to be output to the subsequent stage when the new difference value absolute value temporarily stores a difference value absolute value z ₀ to z ₆ which are sequentially input is inputted ~d ₆ , multiplication units m _{0 to} m ₆ that multiply the difference value absolute values z _{0 to} z ₆ by filter coefficients f _{0 to} f _6, and addition units a _{1 to} a ₆ that sum the multiplied values. A division unit div for dividing the obtained value by the divisor.

The delay units d _{1 to} d ₆ are connected in series. The difference value absolute value z ₀ input to the delay unit d1 is also input to the multiplication unit m ₀ . Delay unit d ₁ to d each difference value absolute value z ₁ to z ₅ stored in each of the ₅ input to each of the delay units d ₁ to d delay portion d ₂ is output from each of the ₅ to d ₆ And is also input to each of the multipliers m _{1 to} m ₅ . Also, the difference value absolute value z ₆ stored in the delay unit d ₆ is input to the multiplier m ₆ are outputted from the delay section d _6. The difference value absolute values z _{0 to} z ₆ are obtained from pixels aligned in the horizontal direction.

The multipliers m _{0 to} m ₆ multiply the input difference value absolute values z _{0 to} z ₆ by filter coefficients f _{0 to} f ₆ , respectively. The values after multiplication are input to the addition units a _{1 to} a ₆ and added together, and input to the division unit div. In the division unit div, the value after the addition is divided by a predetermined divisor, and the amount of motion of the pixels (for 7 pixels) aligned in a line in the horizontal direction is calculated.

  In addition, when calculating | requiring a motion amount based on the difference value absolute value of 35 pixels as mentioned above, a motion amount is calculated by using the same method with respect to the difference value absolute value of the pixels for five rows. . For example, all values obtained by multiplying the absolute values of the difference values of 35 pixels by the filter coefficient are added together by the adding unit, and the value obtained by the addition is a divisor (5 rows = the total value of 35 filter coefficients) ) To get the amount of movement.

<First Configuration Example of Filter Coefficient Calculation Unit>
A first configuration example of the filter coefficient calculation units 624a, 624b, 624c1, and 624c2 will be described with reference to the drawings. FIG. 14 is a block diagram illustrating a first configuration example of the filter coefficient calculation unit. In addition, the filter coefficient calculation unit of the present configuration example is configured using the concept of the ε-filter.

  FIG. 14 may show a part of the filter coefficient calculation unit. FIG. 14 shows a configuration for calculating a filter coefficient that multiplies the pixel values of one row of pixels (including the target pixel) aligned in the horizontal direction. Therefore, in the case of calculating a filter coefficient for multiplying pixels aligned over a plurality of rows, FIG. 14 shows a part of the filter coefficient calculation unit. In the following configuration, the filter coefficient calculation unit calculates a filter coefficient for multiplying 35 pixels (5 rows in the vertical direction and 7 columns in the horizontal direction) including the target pixel and surrounding pixels centered on the target pixel. A case will be described.

As illustrated in FIG. 14, the filter coefficient calculation unit (part) temporarily stores sequentially input pixel values p _{0 to} p ₆ and outputs to the subsequent stage when a new pixel value is input. a Department e ₁ to e _6, a subtracting section s ₀ ~s ₆ for obtaining a difference value between the pixel value p ₃ of a pixel of interest and the pixel value p ₀ ~p ₆ input, output from the subtraction unit s ₀ ~s ₆ Comparison units c _{0 to} c ₆ that compare the difference value to a predetermined value and output a filter coefficient based on the comparison result, and addition units b _{1 to} b ₆ that sum the outputs of the comparison unit. .

The delay units e _{1 to} e ₆ are connected in series. The pixel value p ₀ input to the delay unit e ₁ is also input to the subtraction unit s ₀ . Each pixel value p ₁ ~p ₅ stored in each of the delay units e ₁ to e _5, is input to each of the delay units e ₁ delay portion is output from each of ~e ₅ e ₂ ~e ₆ At the same time, it is also input to each of the subtraction units s _{1 to} s ₅ . Further, the pixel value p ₆ stored in the delay unit e ₆ is input to the subtraction unit s ₆ are outputted from the delay section e _6. The pixel value p ₀ ~p ₆ are values of the pixels aligned in a row in the horizontal direction. Further, the pixel value p ₃ is the pixel value of the target pixel.

The subtraction units s _{0 to} s ₆ calculate difference values between the input pixel values p _{0 to} p ₆ and the pixel value p ₃ of the target pixel, respectively. For example, the difference value is calculated by subtracting the pixel value p ₃ of the target pixel from each of the pixel values p _{0 to} p ₆ . The obtained difference value is input to each of the comparison units c _{0 to} c ₆ and compared with the threshold value T. In particular, it is determined whether or not the absolute value of the difference value is equal to or less than a threshold value T.

Here, as described above, when a value of 1 or 0 is calculated as the filter coefficient, the following equation (3) is obtained. In the following formula (3), _pk is a pixel value of an arbitrary pixel, and _pc is a pixel value of a target pixel. As shown in the following equation (3), the filter coefficient = 1 is output from the comparison units c _{0 to} c ₆ determined to be equal to or less than the threshold value T. On the other hand, the filter coefficient = 0 is output from the comparison units c _{0 to} c ₆ determined to be larger than the threshold T.

Each value output from the comparator c ₀ to c ₆ are summed is input to the addition unit b ₁ ~b _6. The value obtained by the addition is a divisor of pixels (for 7 pixels) aligned in the horizontal direction. When the filter coefficient is calculated for the 35 pixels as described above, for example, a value obtained by adding the divisors for 5 rows obtained in the same manner is a divisor output from the filter coefficient calculation unit. Become.

  If the filter coefficient is calculated using the concept of ε-filter and the amount of motion is calculated as in this configuration example, the amount of motion can be calculated with high accuracy using a simple configuration. .

FIG. 15 shows a configuration for calculating a filter coefficient for multiplying the pixel value of pixels aligned in the horizontal direction (not including the target pixel). The configuration shown in FIG. 15 is substantially the same as the configuration shown in FIG. Specifically, the delay units e _{11 to} e ₁₆ , the subtraction units s _{10 to} s ₁₆ , the comparison units c _{10 to} c ₁₆ , and the addition units b _{11 to} b ₁₆ in FIG. 15 are the delay units e _{1 to} e in FIG. ₆ , the subtraction units s _{0 to} s ₆ , the comparison units c _{0 to} c ₆ , and the addition units b _{1 to} b ₆ , respectively. However, the configuration illustrated in FIG. 15 is different from the configuration illustrated in FIG. 14 in that the pixel value p ₃ of the target pixel is input from the outside and input to each of the subtraction units s _{10 to} s ₁₆ .

  Further, as described above, the comparison is made when the filter coefficient is set to a value of 0 or more and 1 or less, and the pixel value having a large difference from the pixel value of the target pixel is set to be a smaller value. FIG. 16 shows an output example of the unit. FIG. 16 is a graph illustrating an output example of the comparison unit.

In the example shown in FIG. 16, the absolute value of the difference value between the pixel value p _c of the pixel of interest and the pixel value p _k of a pixel is, the filter coefficient when it is below the threshold T is constant at 1. The absolute value of the difference value between the pixel value p _c of the pixel of interest and the pixel value p _k of a pixel is decreased as greater than the threshold T, it approaches zero. In addition, although FIG. 16 shows the case of linearly decreasing, it may be decreased nonlinearly.

  If comprised in this way, it will become possible to calculate a motion amount precisely. For this reason, it is possible to calculate a more accurate amount of motion.

<Second Configuration Example of Filter Coefficient Calculation Unit>
Next, a second configuration example of the filter coefficient calculation units 624a, 624b, 624c1, and 624c2 will be described with reference to the drawings. FIG. 17 is a block diagram illustrating a second configuration example of the filter coefficient calculation unit, and corresponds to FIG. 14 illustrating the first configuration example. In addition, parts that are the same as those in the first configuration example are given the same reference numerals, and detailed descriptions thereof are omitted.

  FIG. 17 may show a part of the filter coefficient calculation unit. FIG. 17 shows a configuration for calculating a filter coefficient for multiplying a pixel value of one row of pixels (including a target pixel) aligned in the horizontal direction. Therefore, in the case of a configuration for calculating a filter coefficient for multiplying pixels aligned over a plurality of rows, FIG. 17 shows a part of the filter coefficient calculation unit. In the following configuration, the filter coefficient calculation unit calculates filter coefficients for multiplying 35 pixels (5 rows in the vertical direction and 7 columns in the horizontal direction) including the target pixel and the surrounding pixels centered on the target pixel. A case will be described.

As illustrated in FIG. 17, the filter coefficient calculation unit (part) includes delay units e _{1 to} e ₆ , subtraction units s _{0 to} s ₆ , comparison units c _{0 to} c ₆ , and addition units b _{1 to} b. ₆ and a smoothing unit SM that smoothes the pixel value p ₃ of the target pixel and the pixel values p ₂ and p _{4 of} the neighboring pixels. Further, the smoothing unit SM adds the addition unit g ₀ , g ₁ that adds the pixel value p ₃ of the target pixel and the pixel values p ₂ , p _{4 of the} neighboring pixels, and the addition unit g ₀ , g ₁ . A division unit h that divides the obtained value by the number of added pixels.

Similarly to the first configuration example, the delay units e _{1 to} e ₆ are connected in series. The delay unit e pixel value p ₀ to be input to ₁ also input to the subtraction unit s _0, each pixel value p ₁ ~p ₅ stored in each of the delay units e ₁ to e ₅ delay portion e ₂ is input to each of the to e ₆ to each of the subtraction unit s ₁ ~s ₅ is input, the pixel values p ₆ stored in the delay unit e ₆ is inputted to the subtraction unit s _6. However, the pixel value p ₃ of the target pixel and the pixel values p ₂ and p ₄ of the neighboring pixels are output from the delay unit e ₂ and the delay units e ₁ and e ₃ , respectively, and input to the smoothing unit SM.

In the smoothing unit SM, first, the pixel values p _{2 to} p ₄ input by the adding units g ₀ and g ₁ are added together. Then, the value obtained by the addition is divided by the division unit h. In this example, three pixel value p ₂ ~p ₄ divided by 3 for smoothing to obtain the pixel values p _a of the smoothed. In FIG. 17, the number of pixel values to be smoothed is 3 for the sake of concrete description, but other numbers may be used.

Subtraction unit s ₀ ~s ₆ includes a pixel value p ₀ ~p ₆ respectively input, calculates a difference value between the pixel value p _a. For example, to calculate a difference value by subtracting the pixel values p _a of each pixel value p ₀ ~p _6. Each value after the subtraction is input to each of the comparison units c _{0 to} c ₆ and compared with the threshold value T. In particular, it is determined whether or not the absolute value of the difference value is equal to or less than a threshold value T.

The comparison units c _{0 to} c ₆ perform the same operation as in the first configuration example. For example, in the case of a configuration in which a value of 1 or 0 is calculated as the filter coefficient, determination is made as shown in the above equation (3) (however, the pixel value _pc of the target pixel in equation (3) is smoothed) changing the pixel values p _a later reduction). Then, the filter coefficients f _{0 to} f ₆ are output from the comparison units c _{0 to} c _6, respectively.

Each value output from the comparator c ₀ to c ₆ are summed is input to the addition unit b ₁ ~b _6. The value obtained by the addition is a divisor of pixels (for 7 pixels) aligned in the horizontal direction. When the filter coefficient is calculated for the 35 pixels as described above, for example, a value obtained by adding the divisors for 5 rows obtained in the same manner is a divisor output from the filter coefficient calculation unit. Become. In this case, in order to calculate the pixel value p _a of the smoothed, it may be configured as the use of the pixel values of the pixels of the plurality of rows further.

In this configuration example, obtains the pixel values p _a with not only the pixel value of the pixel of interest is also the pixel values of neighboring pixels, calculates a filter coefficient according to the difference value between each pixel value and the pixel value p _a To do. For this reason, even if the pixel value of the target pixel becomes an abnormal value due to the influence of noise or the like, it is possible to obtain the motion amount with high accuracy. Therefore, it is possible to effectively reduce noise and to suppress the occurrence of tailing and afterimages.

Also in this configuration example, the filter coefficient may be set to a value of 0 or more and 1 or less, and may be set so that a value that is multiplied by a pixel value having a large difference from the pixel value of the target pixel becomes a smaller value. Absent. That is, the comparison units c _{0 to} c ₆ may output filter coefficients as shown in FIG. 16, for example.

<Third Configuration Example of Filter Coefficient Calculation Unit>
Next, a third configuration example of the filter coefficient calculation units 624a, 624b, 624c1, and 624c2 will be described with reference to the drawings. FIG. 18 is a block diagram illustrating a third configuration example of the filter coefficient calculation unit, and corresponds to FIG. 14 illustrating the first configuration example. In addition, parts that are the same as those in the first configuration example are given the same reference numerals, and detailed descriptions thereof are omitted. In addition, the filter coefficient calculation unit of this configuration example has a configuration using the concept of a bilateral filter.

  FIG. 18 may show a part of the filter coefficient calculation unit. In particular, FIG. 18 shows a configuration for calculating a filter coefficient that multiplies the pixel values of pixels (including a target pixel) for one row aligned in the horizontal direction. Therefore, in the case of calculating a filter coefficient for multiplying pixels aligned over a plurality of rows, FIG. 18 shows a part of the filter coefficient calculation unit. In the following configuration, the filter coefficient calculation unit calculates a filter coefficient for multiplying 35 pixels (5 rows in the vertical direction and 7 columns in the horizontal direction) including the target pixel and surrounding pixels centered on the target pixel. A case will be described.

As illustrated in FIG. 18, the filter coefficient calculation unit (a part) adjusts the difference values output from the delay units e _{1 to} e ₆ , the subtraction units s _{0 to} s _6, and the subtraction units s _{0 to} s _6. Computation units q _{0 to} q ₆ that calculate filter coefficients using the use coefficient σ _r and addition units b _{1 to} b ₆ that sum the outputs of the computation units q _{0 to} q ₆ are provided.

Similarly to the first configuration example, the delay units e _{1 to} e ₆ are connected in series. The delay unit e pixel value p ₀ to be input to ₁ also input to the subtraction unit s _0, each pixel value p ₁ ~p ₅ stored in each of the delay units e ₁ to e ₅ delay portion e ₂ is input to each of the to e ₆ to each of the subtraction unit s ₁ ~s ₅ is input, the pixel values p ₆ stored in the delay unit e ₆ is inputted to the subtraction unit s _6. In addition, the pixel value of the pixel of interest becomes p _3.

The subtraction units s _{0 to} s ₆ calculate difference values between the input pixel values p _{0 to} p ₆ and the pixel value p ₃ of the target pixel, respectively. For example, the difference value is calculated by subtracting the pixel value p ₃ of the target pixel from each of the pixel values p _{0 to} p ₆ . The obtained difference value is input to each of the arithmetic units q _{0 to} q ₆ . In addition, an adjustment coefficient σ _r necessary for calculation is input to the calculation units q _{0 to} q ₆ .

The operation of the arithmetic units q _{0 to} q ₆ will be described with reference to the drawings. FIG. 19 is a schematic diagram for explaining the operation of the calculation unit. In the following, for simplification of description, the pixel values are re-assigned. In particular, the pixel value p ₃ of the pixel of interest is set to the pixel value P _18, and the pixel values p _{0 to} p ₆ are set to P _{15 to} P ₂₁ . Further, the pixels of each row to which the pixel of interest does not contain, from above, and _{P 1 ~P 7, P 8 ~P} 14, P 22 ~P 28, P 29 ~P 35.

FIG. 19A shows the pixel values P _{1 to} P ₃₅ input to the respective calculation units. Note that pixel values P _{15 to} P ₂₁ are input to the above-described arithmetic units q _{0 to} q ₆ , respectively. FIG. 19B shows domain filter coefficients D _{1 to} D ₃₅ corresponding to the pixel values P _{1 to} P ₃₅ , respectively. FIG. 19C shows range filter coefficients R _{1 to} R ₃₅ corresponding to the pixel values P _{1 to} P ₃₅ , respectively.

The domain filter coefficient D _i is a weight set based on the position of the target pixel and the position of the pixel input to the calculation unit. Therefore, the value is determined for each calculation unit. This domain filter coefficient D _i is set as shown in the following equation (4). In the following equation (4), x _i and y _i indicate the positions of pixels having pixel values input to the calculation unit, and x ₁₈ and y ₁₈ indicate the positions of the target pixel. Σ _d is a coefficient for adjusting the value of the domain filter coefficient D _i . The domain filter coefficient D _i may be recorded in each calculation unit.

The range filter coefficient R _i is a weight set based on a difference value between the pixel value of the target pixel and the pixel value of the pixel input to the calculation unit. This range filter coefficient R _i is set as shown in the following equation (5). In the following formula (5), P _i is a pixel value input to the calculation unit, and P ₁₈ is a pixel value of the target pixel. Σ _r is a coefficient for adjusting the value of the range filter coefficient R _i .

The arithmetic unit outputs a value obtained by multiplying the domain filter coefficient D _i and the range filter coefficient R _i , that is, D _i × R _i . This output value becomes the filter coefficient. In particular, filter coefficients f _{0 to} f ₆ (D ₁₅ × R _{15 to} D ₂₁ × R ₂₁ ) are output from the arithmetic units q _{0 to} q ₆ .

The filter coefficients f ₀ ~f ₆ output from the arithmetic unit q ₀ to q ₆ are summed is input to the addition unit b ₁ ~b _6, as the divisor of the pixels aligned in the horizontal direction (seven minutes) Input to the filter section. When the filter coefficient is calculated for the 35 pixels as described above, for example, a value obtained by adding the divisors for 5 rows obtained in the same manner is a divisor output from the filter coefficient calculation unit. Become. When pixels in a row not including the target pixel are input, the pixel value p ₃ of the target pixel may be input from another as in FIG.

In the case of this configuration example, the motion amount M output from the entire motion amount calculation unit is expressed by the following equation (6). In the following equation (6), P _ci is the pixel value of the i th pixel in the input image of the current frame, and P _pi is the pixel value of the i th pixel in the output image of the previous frame.

  If the filter coefficient is calculated using the concept of the bilateral filter and the motion amount is calculated as in this configuration example, the motion amount can be calculated with higher accuracy.

Also in this configuration example, similarly to the second configuration example, instead of using the pixel value p ₃ of a pixel of interest, it may be possible to use a pixel value p _a of performing smoothing.

<Other configuration examples>
About the 1st-3rd structural example, it is good also as a structure by which the amount of motion output is suppressed, so that a divisor is small. For example, in the first configuration example, if the divisor is 1 (the number of pixels with filter coefficient = 1), the calculated motion amount may be reduced to 1/16 and output. Similarly, if the divisor is 2, the amount of movement is reduced to 1/8, if the divisor is 3, the amount of movement is reduced to 1/4, and if the divisor is 4, the amount of movement is reduced to 1/2. If the divisor is 5 or more, the calculated motion amount may be output as it is.

  The case where the divisor is small indicates that pixels having pixel values close to the pixel value of the target pixel exist only in a small area. That is, the pixel value of the pixel is highly likely to be affected by noise. However, if the divisor is small, the influence of the pixel value of this pixel increases, and the amount of motion increases. Then, the removal value is reduced without being determined to be stationary, and noise is not sufficiently reduced.

  By adopting the configuration of this configuration example, it is possible to suppress the case where the divisor is small and the amount of motion is large. And it becomes possible to reduce the generated noise effectively.

  Further, the above-described motion amount calculation and noise reduction processing can be used, for example, when a moving image is captured or reproduced, or when a still image is captured. However, when used when capturing a still image, a plurality of frames of images are acquired at the time of imaging.

  In the imaging apparatus 1 according to the embodiment of the present invention, a control device such as a microcomputer may perform the operations of the image processing unit 6, the noise reduction processing unit 60, the motion amount calculation units 62 and 62a to 62c, and the like. I do not care. Further, all or part of the functions realized by such a control device are described as a program, and the program is executed on a program execution device (for example, a computer) to realize all or part of the functions. It doesn't matter if you do.

  In addition to the above-described case, the imaging apparatus 1 and the noise reduction processing unit 60 in FIG. 1, the motion amount calculation unit 62 in FIG. 2, the motion amount calculation unit 62a in FIG. 6, the motion amount calculation unit 62b in FIG. 11 motion amount calculation unit 62c, the filter unit in FIG. 13, and the filter coefficient calculation unit in FIGS. 14, 15, 17, and 18 can be realized by hardware or a combination of hardware and software. Further, when the imaging apparatus 1, the motion amount calculation units 62, 62a to 62c, the filter unit, and the filter coefficient calculation unit are configured using software, a block diagram of a part realized by software is a functional block diagram of the part Is represented.

  As mentioned above, although each embodiment of the present invention was described, the scope of the present invention is not limited to this, and can be executed with various modifications without departing from the gist of the invention.

  The present invention can be applied to an imaging device that performs noise reduction processing on an image obtained by imaging, and various imaging devices such as a digital still camera that captures a still image and a digital video that captures a moving image. It is applicable to.

These are block diagrams which show the structure of the imaging device in embodiment of this invention. These are block diagrams which show the structure of the noise reduction process part with which the imaging device in embodiment of this invention is equipped. These are the schematic diagrams which show the structure of a noise removal amount calculation part. These are the graphs which show an example of the setting method of noise removal amount. These are the graphs which show an example of the setting method of a feedback coefficient. These are the schematic diagrams which show the structure of 1st Example of a motion amount calculation part. These are the schematic diagrams which show the operation example of 1st Example of a motion amount calculation part. These are the schematic diagrams which show the structure of 2nd Example of a motion amount calculation part. These are the schematic diagrams which show the operation example of 2nd Example of a motion amount calculation part. These are the schematic diagrams which show the operation example of 2nd Example of a motion amount calculation part. These are the schematic diagrams which show the structure of 3rd Example of a motion amount calculation part. These are the schematic diagrams which show the operation example of 3rd Example of a motion amount calculation part. These are block diagrams shown about the structural example of a filter part. These are block diagrams which show the 1st structural example of a filter coefficient calculation part. These are block diagrams which show the 1st structural example of a filter coefficient calculation part. These are the graphs which show the example of an output of a comparison part. These are block diagrams which show the 2nd structural example of a filter coefficient calculation part. These are block diagrams which show the 3rd structural example of a filter coefficient calculation part. These are the schematic diagrams explaining operation | movement of a calculating part. These are the schematic diagrams which show the structure of the motion amount calculation part mounted in the conventional noise reduction processing apparatus. These are the schematic diagrams which show the operation example of the motion amount calculation part mounted in the conventional noise reduction processing apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Image sensor 3 Lens part 4 AFE
5 microphone 6 image processing unit 60 noise reduction processing unit 61 noise removal amount calculation unit 61a difference value calculation unit 61b noise removal amount setting unit 62, 62a to 62c motion amount calculation unit 621 difference value calculation unit 622 absolute value calculation unit 623, 623c Filter unit 624a, 624b, 624c1, 624c2 Filter coefficient calculation unit 63 Feedback coefficient calculation unit 64 Noise reduction unit 64a Multiplication unit 64b Subtraction unit 7 Audio processing unit 8 Compression processing unit 9 Driver unit 10 External memory 11 Decompression processing unit 12 Image output circuit Unit 13 Audio output circuit unit 14 CPU
15 Memory 16 Operation section 17 TG section 18 Bus 19 Bus

Claims (8)

In an imaging device comprising: an imaging unit that captures an image and a noise reduction processing unit that obtains an output image by performing noise reduction processing on the image obtained by the imaging unit;
The noise reduction processing unit
A motion amount calculation unit that calculates a motion amount based on a difference value between pixel values of corresponding pixels of the first image and the second image captured at different timings captured by the imaging unit;
A noise removing unit that obtains an output image by subtracting a removal value from a pixel value of a pixel of the first image,
The motion amount calculation unit calculates the motion amount based on a value obtained by adding a weight to a difference value between pixel values of corresponding pixels of the first image and the second image, Setting the weight based on the first image or the second image;
The imaging apparatus, wherein the noise removing unit sets the removal value based on the amount of motion. The removal value is calculated based on a difference value between pixel values of corresponding pixels of the first image and the second image, and is set to be a smaller value as the amount of motion increases.
The motion amount calculation unit divides a value obtained by adding and adding the weight to a difference value of pixel values of corresponding pixels of the first image and the second image by a value obtained by adding the weight, The imaging apparatus according to claim 1, wherein the movement amount is calculated. The second image is an output image that has been captured before the first image and has been subjected to the noise reduction processing,
The imaging device according to claim 1, wherein the motion amount calculation unit sets the weight based on the second image and calculates the motion amount.   The motion amount calculation unit selects the weight that is closer to a predetermined condition from the weight set based on the first image and the weight set based on the second image, and The imaging apparatus according to claim 1, wherein the amount of motion is calculated using a weight. The motion amount calculation unit calculates the motion amount based on a pixel value of each pixel of a pixel group including a target pixel of the first image or the second image and a peripheral pixel of the target pixel. And
2. The weight is set so that a pixel having a smaller pixel value with respect to a pixel value of the target pixel among the respective pixels included in the pixel group is set to be larger. Item 5. The imaging device according to any one of Items 4 to 5. The motion amount calculation unit calculates the motion amount based on a pixel value of each pixel of a pixel group including a target pixel of the first image or the second image and a peripheral pixel of the target pixel. And
Among the pixels included in the pixel group, the weight is set to be larger as the pixel has a smaller pixel value than the value obtained by smoothing the pixel value of the pixel of interest and its neighboring pixels. The imaging apparatus according to any one of claims 1 to 4, wherein the imaging apparatus is characterized. The motion amount calculation unit calculates the motion amount based on a pixel value of each pixel of a pixel group including a target pixel of the first image or the second image and a peripheral pixel of the target pixel. And
The movement amount calculation unit is obtained based on a first component obtained based on the position of the target pixel and the position of the peripheral pixel, and a difference value between the pixel value of the target pixel and the pixel value of the peripheral pixel. The imaging device according to claim 1, wherein the second component is calculated, and a value obtained by multiplying the first component and the second component is used as the weight.   The imaging apparatus according to claim 1, wherein the motion amount calculation unit sets the motion amount to be smaller as a value obtained by adding the weights is smaller.

Images