JP2008205737A - Imaging system, image processing program, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging system etc., capable of obtaining a smoothing model with an edge preserved therein by operational processing with a comparatively small load. <P>SOLUTION: The imaging system includes: a difference calculating part 203 for calculating a pixel value divergence degree between a target pixel in a target area and a peripheral pixel; a distance calculating part 202 for calculating a position divergence degree between the target pixel and the peripheral pixel; a peripheral pixel coefficient calculating part 207 for obtaining a correlative relation value by multiplying the pixel value divergence degree by the position divergence degree, obtaining a correlative index value by adding a tuning value to the correlative relation value and obtaining the reciprocal number, and obtaining the filter coefficient of the peripheral pixel from the correlative index value; a target pixel coefficient calculating part 208 for obtaining the filter coefficient of the target pixel from the pixel value divergence degree; and a convolution calculating part 210 for calculating the smoothing model pixel value of the target pixel by performing filter calculation by the use of the obtained filter coefficient. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging system, an image processing program, and an image processing method for calculating a smoothing model used for image noise reduction.

  An image signal obtained from an imaging system generally contains a noise component. This noise is generated due to random noise generated from an image sensor or an analog circuit, a defective pixel of the image sensor, and the like. It can be roughly divided into spike noise.

  Among them, random noise has characteristics close to white noise characteristics, and various noise models have been proposed.

  Various techniques for reducing such random noise have been proposed.

に示すようなフィルタを用いるものである。このＳＵＳＡＮフィルタは、数式を見れば分かるように、画素位置（ｘ，ｙ）の注目画素と画素位置（ｘ＋ｉ，ｙ＋ｉ）の各周辺画素との距離と、注目画素の画素値ｆ（ｘ，ｙ）と各周辺画素ｆ（ｘ＋ｉ，ｙ＋ｉ）の画素値との差分と、に基づいて、相関性の高い（すなわち、距離が小さいほど相関性が高く、かつ差分が小さいほど相関性が高い）画素は重みを大きく、相関性の低い（すなわち、距離が大きいほど相関性が低く、かつ差分が大きいほど相関性が低い）画素は重みを小さくするフィルタとなっている。そして、このときに適用される重みは、注目画素と周辺画素との距離に応じたガウス分布と、注目画素値と周辺画素値との差分に応じたガウス分布と、の積となっている。 For example, a first technique known as a SUSAN filter (bilateral filter) (see [Background Art] in Japanese Patent Application Laid-Open No. 2004-159511 described later) has the following formula:

A filter as shown in FIG. As can be seen from the mathematical expression, this SUSAN filter can detect the distance between the pixel of interest at the pixel position (x, y) and each peripheral pixel at the pixel position (x + i, y + i), and the pixel value f (x, y) of the pixel of interest. ) And the difference between the pixel values of the surrounding pixels f (x + i, y + i) and the higher the correlation (that is, the smaller the distance, the higher the correlation, and the smaller the difference, the higher the correlation). The pixel has a large weight and a low correlation (that is, the correlation is low as the distance is large and the correlation is low as the difference is large) is a filter that reduces the weight. The weight applied at this time is a product of a Gaussian distribution according to the distance between the target pixel and the peripheral pixels and a Gaussian distribution according to the difference between the target pixel value and the peripheral pixel value.

に示すようなフィルタを用いるものである。この技術は、画像ブロック毎に分布係数（ここではＵ（Ｘ，Ｙ））を算出して、画像ブロック毎にフィルタ係数を制御するものとなっている。 Moreover, as a 2nd technique, the technique as described in Unexamined-Japanese-Patent No. 2004-159511 is mentioned, for example. The technique described in the publication is obtained by improving the above-described SUSAN filter,

A filter as shown in FIG. In this technique, a distribution coefficient (here, U (X, Y)) is calculated for each image block, and a filter coefficient is controlled for each image block.

に示すように、ノイズ低減後の注目画素濃度値Ｓ0を算出する技術が記載されている。該公報に記載の技術は、図１６（Ａ）〜図１６（Ｃ）に示すように、周辺濃度値Ｕと注目画素濃度値Ｓとの差分が所定の設定値αよりも大きい場合には周辺濃度値Ｕに適用するフィルタ係数Ｗuを相対的に大きくし、逆に、該差分が所定の設定値αよりも小さい場合には注目画素濃度値Ｓに適用するフィルタ係数Ｗsを相対的に大きくする（つまり、注目画素と周辺画素との重みの比率を変化させる）ものとなっている。なお、図１６におけるＩＤは、使用するＬＵＴに対応するＩＤ番号であり、周辺濃度値Ｕに応じて決定される。ここに、図１６は、従来のフィルタリング処理における注目画素と周辺画素との重み係数の相関を概念的に示す図である。 On the other hand, as a third technique, for example, Japanese Patent Application Laid-Open No. 10-200271 discloses a peripheral density value U representatively representing density values of peripheral pixels other than the target pixel in the target area (in the embodiment of the publication). An average pixel value) is calculated, and a filter coefficient (load value) Ws applied to the target pixel based on the difference between the peripheral density value U and the target pixel density value S, and a filter coefficient (load value) applied to the peripheral pixel ) Determine Wu and the following formula:

As shown in FIG. 5, a technique for calculating a target pixel density value S0 after noise reduction is described. As shown in FIGS. 16 (A) to 16 (C), the technique described in this publication is used when the difference between the peripheral density value U and the target pixel density value S is larger than a predetermined set value α. The filter coefficient Wu applied to the density value U is relatively increased. Conversely, when the difference is smaller than the predetermined set value α, the filter coefficient Ws applied to the target pixel density value S is relatively increased. (In other words, the weight ratio between the target pixel and the surrounding pixels is changed). The ID in FIG. 16 is an ID number corresponding to the LUT to be used, and is determined according to the peripheral density value U. FIG. 16 is a diagram conceptually showing the correlation between the weighting factors of the pixel of interest and the surrounding pixels in the conventional filtering process.

As a fourth technique, for example, in Japanese Patent Application Laid-Open No. 2004-88149, a noise model is estimated based on factors such as a signal level, a temperature of a CCD, an exposure time, a gain, and a photographing situation. Describes a technique for reducing noise by correcting video to match the shooting scene and suppressing video signal components below the corrected noise model. As shown in FIG. 17, the fourth technique stores video signal components that exceed the noise model, and thus can realize a smoothing process that stores edges as original signals. FIG. 17 is a diagram conceptually showing how the noise at the edge portion remains in the conventional coring process.
JP 2004-159511 A Japanese Patent Laid-Open No. 10-200751 JP 2004-88149 A

  However, since the first technique and the second technique as described above perform processing such as exponential function calculation, and the load of the calculation processing is large, the first technique and the second technique are random for mounting in an imaging system such as a camera. Realizing hardware as a noise reduction device is not practical.

  In addition, since the third technique as described above smoothes all peripheral pixels using the same filter coefficient, it cannot be said that the preservation of edges and fine regions is sufficient.

  And since the 4th technique as mentioned above assumes the smoothing model which took the simple average of the neighborhood pixel, as shown in Drawing 17, the smoothing model is lost at the edge part, The noise reduction effect is insufficient at the edge portion.

  The present invention has been made in view of the above circumstances, and provides an imaging system, an image processing program, and an image processing method that can obtain a smoothed model in which edges are preserved by arithmetic processing with a relatively small load. It is aimed.

  In order to achieve the above object, an imaging system according to a first invention is a target for performing noise reduction processing on a video signal in an imaging system that performs noise reduction processing on a video signal captured from the imaging system. The correlation value calculated based on at least one of the spatial correlation and the pixel value correlation between the target pixel and one or more peripheral pixels located in the spatial vicinity of the target pixel is adjusted. A smoothing signal calculation unit that calculates a smoothing signal for the target pixel based on the adjustment value for the target pixel, and the target pixel is included based on the smoothed signal of the target pixel calculated by the smoothing signal calculation unit A noise amount estimating means for estimating a noise amount; a smoothing signal calculated by the smoothed signal calculating means; and a noise amount estimated by the noise amount estimating means. There are those having a noise reduction means for performing noise reduction processing on the target pixel.

  An imaging system according to a second invention is the imaging system according to the first invention, wherein the smoothed signal calculating means is one or more pixels located in a spatial vicinity of the target pixel in the input image signal and the target pixel. Pixel value for calculating a pixel value divergence degree for each peripheral pixel, and a target area setting unit that sets a target area configured by the peripheral pixels of A divergence degree calculating means, a position divergence degree calculating means for calculating, for each peripheral pixel, a position divergence degree indicating the degree of spatial divergence between the target pixel and the peripheral pixel, and the pixel values relating to the same peripheral pixel Correlation-related value calculation means for obtaining, for each peripheral pixel, a correlation-related value that is a value related to the correlation between the pixel of interest and the peripheral pixel by multiplying the degree of divergence and the position divergence degree for each peripheral pixel. And above A correlation index that adds a tuning value, which is a constant for adjustment, to the correlation value and obtains a correlation index value for each peripheral pixel that serves as an index indicating the correlation between the pixel of interest and the peripheral pixel based on the added value Value calculation means, and filter coefficient calculation means for obtaining, for each pixel, filter coefficients for all pixels included in the region of interest based on at least one of the correlation index value and the value obtained in the process of obtaining the correlation index value. Filter calculation means for calculating a smoothed model pixel value related to the target pixel by performing a filter operation on the target region using the filter coefficient.

  Furthermore, the imaging system according to the third invention is the imaging system according to the second invention, wherein the filter coefficient calculation means adds the pixel value divergence degree with respect to all peripheral pixels in the region of interest. Accordingly, the filter coefficient for the target pixel and the filter coefficient for the peripheral pixel are obtained so that the filter coefficient for the target pixel becomes larger relative to the filter coefficient for the peripheral pixel.

  The imaging system according to a fourth invention is the imaging system according to the third invention, wherein the filter coefficient calculation means adds the pixel value divergence degrees for all the surrounding pixels in the region of interest, The filter coefficient for the pixel of interest is increased within a range of a predetermined upper limit value of 0 or more and 1 or less, and the value obtained by subtracting the filter coefficient for the pixel of interest from 1 is included as a filter coefficient factor for the surrounding pixels. By doing so, the filter coefficient for the pixel of interest is increased relative to the filter coefficient for the surrounding pixels.

または、次の数式

により算出するものであり、
上記位置乖離度算出手段は、位置乖離度ｒ（ｉ，ｊ）を、次の数式

または、次の数式

により算出するものであり、
上記相関関連値演算手段は、相関関連値を、次の演算

により算出するものであり、
上記相関指標値演算手段は、相関指標値Ｃｏｒ（ｉ，ｊ）を、上記相関関連値と上記チューニング値ｔｕｎｅ（ここに、チューニング値ｔｕｎｅは０以上の実数）とに基づき、次の数式

により算出するものであり、
上記フィルタ係数演算手段は、上記注目画素のフィルタ係数Ｗcenterを、次の数式

により算出する（ここに、総和は、注目領域に含まれる全周辺画素に対応する（ｉ，ｊ）についてとる（すなわち、総和からは、ｉ＝０かつｊ＝０の場合は除外される）ものとする）ものであるとともに、上記周辺画素のフィルタ係数Ｗ（ｉ，ｊ）を、次の数式

により算出する（ここに、総和は、注目領域に含まれる全周辺画素に対応する（ｉ，ｊ）についてとる（すなわち、総和からは、ｉ＝０かつｊ＝０の場合は除外される）ものとする）ものであり、
上記フィルタ演算手段は、注目画素に係る平滑化モデル画素値Ｐ’（ｘ，ｙ）を、次の数式、

により算出する（ここに、総和は、注目領域に含まれる全周辺画素に対応する（ｉ，ｊ）についてとる（すなわち、総和からは、ｉ＝０かつｊ＝０の場合は除外される）ものとする）ものである。 An imaging system according to a fifth invention is the imaging system according to the third invention, wherein the position of the pixel of interest is (x, y) (where x and y are real numbers), and the position of the surrounding pixels is (x + i, y + j) (where i and j are real numbers, i and j are not 0 at the same time), the pixel value of the pixel of interest is P (x, y), and the pixel values of the peripheral pixels are P When (x + i, y + j),
The pixel value divergence degree calculation means calculates the pixel value divergence degree D (i, j) as follows:

Or the following formula

Is calculated by
The position divergence calculating means calculates the position divergence r (i, j) by the following formula.

Or the following formula

Is calculated by
The correlation related value calculating means calculates the correlation related value by

Is calculated by
The correlation index value calculation means calculates the correlation index value Cor (i, j) based on the correlation-related value and the tuning value tune (where the tuning value tune is a real number greater than or equal to 0) as follows:

Is calculated by
The filter coefficient calculation means calculates the filter coefficient Wcenter of the target pixel by the following formula

(Here, the sum is taken for (i, j) corresponding to all peripheral pixels included in the region of interest (that is, excluded from the sum when i = 0 and j = 0)) And the filter coefficient W (i, j) of the surrounding pixels is expressed by the following formula:

(Here, the sum is taken for (i, j) corresponding to all peripheral pixels included in the region of interest (that is, excluded from the sum when i = 0 and j = 0)) And)
The filter calculation means calculates the smoothed model pixel value P ′ (x, y) relating to the target pixel by the following formula:

(Here, the sum is taken for (i, j) corresponding to all peripheral pixels included in the region of interest (that is, excluded from the sum when i = 0 and j = 0)) And).

  An imaging system according to a sixth invention is the imaging system according to the second invention, wherein the attention area setting means sequentially sets the attention area so that the pixel position of the attention pixel is sequentially changed. A region similarity determining unit that determines whether or not the region of interest currently set by the region setting unit is similar to the region of interest set in the past by the region of interest setting unit; If it is determined that they are similar, the processing of the pixel value deviation degree calculating means, the position deviation degree calculating means, the correlation related value calculating means, the correlation index value calculating means, and the filter coefficient calculating means is omitted. , The filter coefficient already obtained for the attention area set in the past is used for the attention area currently set to the filter calculation means. Ri and calculates the smoothed model pixel values.

  An imaging system according to a seventh invention is the imaging system according to the second invention, wherein the pixel value of the target pixel related to the input image signal and the smoothed model pixel value related to the target pixel are used. The apparatus further includes coring calculation means for performing a coring process.

  An imaging system according to an eighth invention is the imaging system according to the seventh invention, further comprising tuning adjustment means for changing the tuning value in accordance with the inputted image signal and the image signal after the coring process. It is equipped.

  An imaging system according to a ninth invention is the imaging system according to the eighth invention, wherein an input SN calculation means for calculating an SN ratio of the inputted image signal and an SN ratio of the image signal after the coring process are calculated. A noise-reduced SN calculation unit that performs the tuning adjustment according to the S / N ratio of the input image signal and the S / N ratio of the image signal after the coring process. The value is changed.

  An imaging system according to a tenth invention is the imaging system according to the ninth invention, wherein the input SN calculation means extracts a flat area of the input image signal and calculates an SN ratio of the flat area. And the noise-reduced SN calculation means calculates an SN ratio of the same area as the flat area in the image signal after the coring process.

  An image pickup system according to an eleventh aspect of the invention is the image pickup system according to the second aspect of the invention, further comprising image pickup means for picking up an image and outputting an image signal, and the image signal output from the image pickup means is inputted as described above. Image signal.

  An image processing program according to a twelfth aspect of the invention is an attention area setting for setting an attention area composed of a target pixel and one or more peripheral pixels located in the spatial vicinity of the target pixel in an input image signal. A step of calculating a pixel value divergence degree for each peripheral pixel, and a space between the target pixel and the peripheral pixel, and a step of calculating a pixel value divergence degree indicating a degree of divergence between the target pixel and the peripheral pixel. A position divergence degree calculation step for calculating a position divergence degree for each peripheral pixel, and the pixel value divergence degree and the position divergence degree for the same peripheral pixel are multiplied for each peripheral pixel. As a result, a correlation value calculation step for obtaining a correlation value, which is a value related to the correlation between the target pixel and the peripheral pixel, for each peripheral pixel, and a adjustment constant for the correlation value are selected. A correlation index value calculating step for obtaining, for each peripheral pixel, a correlation index value serving as an index indicating the correlation between the pixel of interest and the surrounding pixels based on the added value, and the correlation index value A filter coefficient calculation step for obtaining, for each pixel, filter coefficients for all pixels included in the region of interest based on at least one of the values obtained in the process of obtaining the correlation index value; and the region of interest using the filter coefficient And a filter calculation step of calculating a smoothed model pixel value related to the target pixel by performing a filter calculation on the program.

  An image processing method according to a thirteenth aspect of the present invention is an attention area setting step of setting an attention area composed of a target pixel and one or more peripheral pixels located in the spatial vicinity of the target pixel in an input image signal; A pixel value divergence calculating step for calculating a pixel value divergence degree indicating the degree of divergence between the target pixel and the peripheral pixel for each peripheral pixel; and a spatial divergence between the target pixel and the peripheral pixel. A position divergence degree calculating step for calculating a position divergence degree for each peripheral pixel, and multiplying the pixel value divergence degree and the position divergence degree for the same peripheral pixel for each peripheral pixel. A correlation value calculation step for obtaining a correlation value, which is a value related to the correlation between the target pixel and the peripheral pixel, for each peripheral pixel, and a tuning value that is an adjustment constant is added to the correlation value. A correlation index value calculating step for obtaining a correlation index value for each peripheral pixel as an index indicating the correlation between the pixel of interest and the peripheral pixel based on the added value; and a process of calculating the correlation index value and the correlation index value A filter coefficient calculation step for obtaining, for each pixel, a filter coefficient for all pixels included in the region of interest based on at least one of the values obtained in step 1, and performing a filter operation on the region of interest using the filter coefficient And a filter operation step of calculating a smoothed model pixel value related to the target pixel.

  According to the imaging system, the image processing program, and the image processing method of the present invention, it is possible to obtain a smoothed model in which edges are stored by an arithmetic processing with a relatively small load.

  Embodiments of the present invention will be described below with reference to the drawings.

[Embodiment 1]
1 to 11 show Embodiment 1 of the present invention, and FIG. 1 is a block diagram showing a configuration of a noise reduction processing unit.

  The noise reduction processing unit 100 constitutes at least a part of the imaging system, and is a smoothing signal calculation unit and a filtering calculation unit 101 as an attention area setting unit, a coring calculation unit 102 as a coring calculation unit, It is comprised.

  An input image signal that is a signal of the input original image is connected to the filtering operation unit 101 and is also connected to the coring operation unit 102. The filtering operation unit 101 is connected to the coring operation unit 102. The coring calculation unit 102 outputs an output signal after noise reduction.

  The filtering operation unit 101 performs smoothing on the input image signal to calculate a smoothed model signal, and outputs the calculated smoothed model signal to the coring operation unit 102.

  The coring calculation unit 102 calculates and outputs an output signal in which noise is suppressed by performing a coring process based on the smoothed model signal input from the filtering calculation unit 101 and the input image signal.

  Next, FIG. 2 is a block diagram illustrating a configuration of the filtering operation unit 101.

  The filtering calculation unit 101 includes a memory 201, a distance calculation unit 202 that is a smoothing signal calculation unit and a position deviation degree calculation unit, a difference calculation unit 203 that is a smoothing signal calculation unit and a pixel value deviation degree calculation unit, A memory 204, a memory 205, a tuning constant storage unit 206, a smoothing signal calculation unit, a correlation-related value calculation unit, a correlation index value calculation unit, a peripheral pixel coefficient calculation unit 207 that also serves as a filter coefficient calculation unit, and smoothing The pixel pixel calculation unit 208 is a signal calculation unit and filter coefficient calculation unit, a memory 209, and a convolution calculation unit 210 that is a smoothing signal calculation unit and filter calculation unit.

  Here, the input original image is stored in the memory 201. The memory 201 is connected to the distance calculation unit 202, the difference calculation unit 203, and the convolution operation unit 210. The distance calculation unit 202 is connected to the peripheral pixel coefficient calculation unit 207 via the memory 204. The difference calculation unit 203 is connected to the surrounding pixel coefficient calculation unit 207 and the target pixel coefficient calculation unit 208 via the memory 205. The tuning constant storage unit 206 is connected to the peripheral pixel coefficient calculation unit 207. The peripheral pixel coefficient calculation unit 207 and the target pixel coefficient calculation unit 208 are connected to the convolution operation unit 210 via the memory 209. A smoothing model signal is output from the convolution operation unit 210.

  Next, the operation of the filtering operation unit 101 configured as shown in FIG. 2 will be described with reference to FIG. FIG. 7 is a flowchart showing the operation of the filtering operation unit 101.

  The input original image is sequentially input to the memory 201 in units of pixels, for example. At this time, the filtering calculation unit 101 does not start the process immediately after the pixel data of the first pixel is input, but delays until the pixel data for a predetermined line that can perform the filtering process is completed.

  Thus, when pixel data for a predetermined line that can be filtered is stored in the memory 201, for example, a pixel in a region of interest as shown in FIG. 4, that is, a pixel position (x, y) (here) In addition, “x” is a coordinate representing the pixel position in the horizontal direction, and “y” is a coordinate representing the pixel position in the vertical direction, although these coordinates can be expressed as real values, May be expressed as an integer value, and hence will be expressed as an integer value in the following.) P (x, y) of the pixel value of the target pixel (hereinafter referred to as “target pixel value” as appropriate) , The surrounding pixel position (x + i, y + j) (where i and j can be expressed as real values, but from a more practical point of view, they will be expressed as integers below. At the same time as j The pixel values of the surrounding pixels (hereinafter referred to as “neighboring pixel values”) P (x + i, y + j) as appropriate are extracted and transferred to the distance calculation unit 202, the difference calculation unit 203, and the convolution calculation unit 210. Is output (step S101). FIG. 4 is a diagram illustrating an example of the relationship between the target pixel and each peripheral pixel in the target region. FIG. 4 shows an example in which the region of interest is 3 × 3 pixels, and −1 ≦ i ≦ 1, −1 ≦ j ≦ 1, but the present invention is not limited to this. It is also possible to extract a region of interest of (positive integer) × (positive integer) pixels.

［数２］

The distance calculation unit 202 determines the degree of spatial divergence indicating the degree of spatial divergence between the pixel of interest (pixel position (x, y)) and each peripheral pixel (pixel position (x + i, y + j)) in the extracted region of interest. r (i, j) is calculated, for example, as a distance as shown in the following formula 1 or as a square of the distance as shown in formula 2 (note that the value calculated by any formula is used as the position divergence degree). Whether r (i, j) is set can be determined as appropriate according to the circuit configuration of the processing system, various conditions relating to the image, and the like.
[Equation 1]

[Equation 2]

  In addition, although the example using the value calculated | required by Numerical formula 1 or Numerical formula 2 as the position deviation degree r (i, j) was shown here, it does not restrict to these. For example, when the pixel is not a square pixel, it is conceivable to multiply i or j by an appropriate coefficient. However, the present invention is not limited to this, and is an amount indicating the degree of spatial divergence between the target pixel and surrounding pixels. If the load at the time of calculation is a relatively small amount, an appropriate amount can be used as the position deviation r (i, j).

  Then, the distance calculation unit 202 outputs the calculated position deviation r (i, j) and stores it in the memory 204 (step S102).

［数４］

On the other hand, the difference calculation unit 203 outputs a pixel value divergence degree D (i, i) indicating the degree of divergence between the pixel value of interest P (x, y) in the extracted attention area and the surrounding pixel values P (x + i, y + j). j) is calculated as, for example, the absolute value of the difference as shown in the following Equation 3 or the square value of the difference as shown in Equation 4.
[Equation 3]

[Equation 4]

  Note that the degree of edge preservation is greater when the square value is used for the pixel value deviation degree D (i, j) than when the absolute value is used for the pixel value deviation degree D (i, j). Therefore, the smoothing strength tends to be small. Therefore, the absolute value is used as the pixel value divergence degree D (i, j) when it is desired to increase the smoothing strength, and the pixel value divergence degree D (i, j) is used when the preservation of edges is more important. It is good to use properly such as using a square value.

  In addition, here, as the pixel value divergence degree D (i, j), the example using the value obtained by Expression 3 or 4 is shown, but the present invention is not limited thereto, and the divergence between the target pixel value and the surrounding pixel value is shown. An appropriate amount can be used as the pixel value deviation degree D (i, j) as long as the load at the time of calculation is a relatively small amount.

  Then, the difference calculation unit 203 outputs the calculated pixel value deviation degree D (i, j) and stores it in the memory 205 (step S103).

ここに、定数Ｃｏｎｓｔは、ΣＤ（ｉ，ｊ）の取り得る最大値などであり、０≦Ｗcenter≦１となるように設定されている。また、総和は、注目領域に含まれる全周辺画素に対応する（ｉ，ｊ）についてとるものとする。従って、総和からは、ｉ＝０かつｊ＝０の場合は除外される。 Subsequently, the pixel-of-interest coefficient calculation unit 208 inputs the pixel value deviation degree D (i, j) from the memory 205, calculates the sum ΣD (i, j) (step S104), and further divides by the constant Const. Thus, the weighting coefficient Wcenter, which is the filter coefficient of the pixel of interest, is calculated as a value proportional to the total sum ΣD (i, j) as shown in the following Expression 5 (step S105).
[Equation 5]

Here, the constant Const is a maximum value that ΣD (i, j) can take, and is set to satisfy 0 ≦ Wcenter ≦ 1. The sum is taken for (i, j) corresponding to all peripheral pixels included in the region of interest. Therefore, it is excluded from the summation when i = 0 and j = 0.

  Further, the target pixel coefficient calculation unit 208 compares the weight coefficient Wcenter calculated as shown in Equation 5 with a preset maximum value max (here, 0 ≦ max ≦ 1) (step S106).

  Then, when the weighting coefficient Wcenter is equal to or less than the maximum value max, the target pixel coefficient calculation unit 208 sets the weighting coefficient calculated by Equation 5 as the final weighting coefficient Wcenter (step S107).

On the other hand, when the weighting coefficient Wcenter calculated by Expression 5 exceeds the maximum value max, the target pixel coefficient calculation unit 208 sets the weighting coefficient Wcenter of the target pixel to the maximum value max as shown in FIG. Clip (step S108). FIG. 5 is a diagram showing how the relationship between the weighting coefficient of the target pixel and the weighting coefficients of the surrounding pixels changes depending on the sum of the pixel value deviation degrees. The process of calculating the weight coefficient Wcenter of the target pixel by the target pixel coefficient calculation unit 208 as described above is expressed by the following Expression 6.
[Equation 6]

  The target pixel coefficient calculation unit 208 stores the calculated weight coefficient Wcenter in the memory 209.

Subsequently, the peripheral pixel coefficient calculation unit 207 inputs the position divergence degree r (i, j) from the memory 204 and also inputs the pixel value divergence degree D (i, j) related to the same peripheral pixel from the memory 205. Then, by multiplying them as shown in the following Expression 7, a correlation-related value that is a value related to the correlation between the target pixel and the peripheral pixel is obtained for each peripheral pixel.
[Equation 7]

Further, the peripheral pixel coefficient calculation unit 207 adjusts the influence of the correlation-related value in the correlation index value by inputting the tuning value tune input from the tuning constant storage unit 206 (where the tuning value tune is a real number of 0 or more. The correlation index value Cor (i, j, which is an index indicating the correlation between the pixel of interest and the surrounding pixels, as shown in the following Expression 8, by adding ) Is calculated for each peripheral pixel (step S109).
[Equation 8]

ここに、総和は、注目領域に含まれる全周辺画素に対応する（ｉ，ｊ）についてとるものとする。従って、総和からは、ｉ＝０かつｊ＝０の場合は除外される。 Further, the peripheral pixel coefficient calculation unit 207 normalizes all correlation index values Cor (i, j) in the attention area, thereby obtaining a weighting coefficient W (i, j) that is a factor of a filter coefficient for the peripheral pixels. Calculation is performed as shown in the following Equation 9.
[Equation 9]

Here, the sum is taken for (i, j) corresponding to all peripheral pixels included in the region of interest. Therefore, it is excluded from the summation when i = 0 and j = 0.

  Then, the surrounding pixel coefficient calculation unit 207 stores the calculated weighting coefficient W (i, j) in the memory 209 (step S110).

ここに、総和は、注目領域に含まれる全周辺画素に対応する（ｉ，ｊ）についてとるものとする。従って、総和からは、ｉ＝０かつｊ＝０の場合は除外される。 The convolution operation unit 210 inputs the target pixel value P (x, y) and each peripheral pixel value P (x + i, y + j) in the target region from the memory 201, and the target pixel weight coefficient Wcenter and each peripheral from the memory 209. A pixel weighting factor W (i, j) is input, and a convolution operation as a filter operation is performed as shown in Equation 10 below, whereby a smoothed model pixel value P ′ (x, y) is calculated (step S111).
[Equation 10]

Here, the sum is taken for (i, j) corresponding to all peripheral pixels included in the region of interest. Therefore, it is excluded from the summation when i = 0 and j = 0.

  As can be seen from Equation 10, the filter coefficient of the pixel of interest is the weight coefficient Wcenter, while the filter coefficient of the surrounding pixels is the weight coefficient W (i, j) multiplied by a factor of (1-Wcenter). Will be. Therefore, as the weight coefficient Wcenter of the target pixel (proportional to the value obtained by adding the pixel value divergence degree D (i, j) to all the surrounding pixels in the target region) increases, the smoothed model pixel value of the target pixel The contribution to P ′ (x, y) increases. On the other hand, since the peripheral pixel includes the factor (1-Wcenter), the smoothed model pixel value P ′ (x, y) of the peripheral pixel increases as the weight coefficient Wcenter of the target pixel increases. ) Will be reduced.

  This is because the weight of the target pixel is increased when the total sum ΣD (i, j) of the pixel value divergence between the target pixel and the surrounding pixels is large, that is, when the target pixel is included in an edge or a fine region. Therefore, smoothing that preserves the value of the target pixel in the input image signal is performed.

  On the other hand, when the sum ΣD (i, j) of pixel value divergence degrees between the target pixel and the peripheral pixels is small, that is, when the target pixel is included in the flat region, the weight of the target pixel is reduced (that is, the peripheral pixel) This means that the smoothing effect by the peripheral pixels works strongly.

  Thus, the present embodiment adaptively changes the weight of the target pixel and the weights of the peripheral pixels.

  Then, the convolution operation unit 210 outputs the calculated smoothed model pixel value P ′ (x, y) to the coring operation unit 102 (step S112).

  The input original image includes noise further included in the original signal obtained from the optical image. The smoothed model signal calculated by the filtering operation unit 101 is as shown in FIG. That is, compared with the smoothing model calculated by taking a simple average of neighboring pixels as shown in FIG. 17, the smoothing model of this embodiment preserves the sharpness of the edge as shown in FIG. Thus, the signal is subjected to smoothing filtering.

  Next, FIG. 3 is a block diagram illustrating a configuration of the coring calculation unit 102.

  The coring calculation unit 102 is configured to include two functional units that are roughly divided into a noise amount estimation unit that is a noise amount estimation unit and a noise reduction processing unit that is a noise reduction unit. First, the former noise amount estimation unit estimates a noise model LUT 301 as a means for storing the relationship between the signal value and the noise amount, an ISO sensitivity storage unit 302 as a means for storing the ISO sensitivity at the time of shooting, and an estimation. And a gain control unit 303 as means for controlling the gain of the noise amount. The latter noise reduction processing unit includes a noise amount correction unit 304 that is a unit that corrects a noise amount based on a user operation, and a coring processing unit 305 that is a unit that performs coring based on the noise amount. Configured.

  The smoothed model signal from the filtering operation unit 101 is input to the noise model LUT 301 and the coring processing unit 305. The input original image is input to the coring processing unit 305. Further, the noise model LUT 301 is connected to the gain control unit 303. The ISO sensitivity storage unit 302 is connected to the gain control unit 303. The gain control unit 303 is connected to the noise amount correction unit 304. The noise amount correction unit 304 is connected to the coring processing unit 305. The coring processing unit 305 outputs an output image after noise reduction.

  The noise model LUT 301 outputs the noise model amplitude Nm to the gain control unit 303 by referring to the lookup table based on the smoothed model pixel value P ′ (x, y) input from the filtering operation unit 101. . The look-up table of the noise model LUT 301 is a table in which the relationship between the signal value and the noise model amplitude Nm is set in advance based on the characteristics of the image sensor that captured the input original image. Here, the signal value and the noise model amplitude Nm are defined by, for example, an average value of pixel values in a predetermined imaging region and a standard deviation when an object with uniform predetermined luminance is imaged.

  The ISO sensitivity storage unit 302 stores the ISO sensitivity at the time of shooting set by the user's operation.

The gain control unit 303 calculates the noise amount NISO by multiplying the noise model amplitude Nm input from the noise model LUT 301 by the gain GISO based on the ISO sensitivity input from the ISO sensitivity storage unit 302. The noise amount NISO is output to the noise amount correction unit 304. This calculation process is shown by the following mathematical formula.
NISO = Nm × GISO

The noise amount correction unit 304 calculates the noise amount N by multiplying the noise amount NISO input from the gain control unit 303 by the gain GU corresponding to the noise reduction intensity set by the user, and calculates the calculated noise. The amount N is output to the coring processing unit 305. This calculation process is shown by the following mathematical formula.
N = NISO × GU

  With this function, for example, the user can arbitrarily set the noise reduction intensity according to the shooting situation (for example, landscape shooting, portrait shooting, night scene shooting, close-up shooting, shooting of a moving subject, etc.) It is possible to adjust the intensity of the reduction.

  FIG. 6 is a diagram showing a state of noise suppression at the edge portion by coring processing, and FIG. 8 is a flowchart showing the operation of the coring processing unit 305.

  As shown in FIG. 6, the coring processing unit 305 uses the smoothed model pixel value P ′ (x, y) as a reference value, and calculates the target pixel value P (x, y) and the reference value of the input image signal. Comparison is made based on the noise amount N output from the noise amount correction unit 304, and coring processing is performed to output a final output signal Pout after noise reduction.

Here, the coring process is performed as follows.
The coring processing unit 305 uses the smoothed model pixel value P ′ (x, y) that is the smoothed model signal input from the filtering operation unit 101 as a reference value, and the target pixel value P (x, y) that is the input image signal. The difference absolute value between y) and the reference value is calculated (step S201), and the calculated difference absolute value is compared with the noise amount N input from the noise amount correction unit 304 (step S202).

  When the difference absolute value is equal to or less than the noise amount N, the coring processing unit 305 replaces the target pixel value P (x, y) with the smoothed model pixel value P ′ (x, y). The final output signal Pout is set (step S203).

  On the other hand, when the difference absolute value is larger than the noise amount N, the coring processing unit 305 determines whether the target pixel value P (x, y) is larger than the smoothed model pixel value P ′ (x, y). Is determined (step S204).

  If the coring processing unit 305 determines that the value is large, the final output signal Pout is obtained by subtracting the noise amount N from the target pixel value P (x, y) (step S206), but is not large ( In other words, the final output signal Pout is obtained by adding the amount of noise N to the target pixel value P (x, y) (step S205).

The calculation process of the final output signal Pout as described above can be expressed by the following formula 11.
[Equation 11]

  Thus, when the process in any of steps S203, S205, and S206 is completed, the final output signal Pout is output (step S207), and this process is terminated.

  According to such processing in the coring calculation unit 102, since the relationship between the signal value and the noise model amplitude Nm is calculated based on the characteristics of the image sensor that captured the input original image, the characteristics of the image sensor. Can be reduced. Further, since the amount of noise corresponding to the smoothed model signal value is estimated for each pixel of the image, it is possible to perform optimum noise reduction from the bright part to the dark part. Furthermore, since the noise amount is corrected according to the ISO sensitivity at the time of shooting, the accuracy of noise amount estimation can be improved. Furthermore, since it is possible for the user to adjust the intensity of noise reduction, it is possible to obtain a noise reduction effect that suits the user's purpose in accordance with shooting conditions and the like.

  Also, in the coring process, since the noise reduction process is performed using the reference value that is a reference for coring and the noise amount that is the range of the coring, the noise reduction process can be effectively performed. Is required to estimate an optimal reference value and noise amount. In contrast, in the present embodiment, since the noise amount is estimated using the smoothed model signal that preserves the sharpness of the edge, the noise amount is accurately estimated even in a steep edge region. It becomes possible. In addition, since the smoothed model signal that preserves the sharpness of the edge is used as the correlating reference value, it is possible to obtain a reference value that preserves the image structure even in a steep edge region. Thereby, according to the coring process of this embodiment, it becomes possible to reduce noise effectively also near the edge.

  FIG. 17 shows a case where a conventional smoothed model signal generated with an average value is used, and becomes a smoothed model signal distorted with respect to a steep edge of the original input image signal. In such a case, the smoothed model pixel value P ′ (x, y) in the vicinity of the steep edge is far from the target pixel value P (x, y) that is the input image signal, and therefore from the noise model LUT 301. The estimated noise amount to be output is not correct. Furthermore, in the vicinity of such a steep edge, the difference absolute value between the smoothed model pixel value P ′ (x, y) calculated in the coring process and the target pixel value P (x, y) that is the input image signal is Since it becomes larger than necessary and is compared with the above-mentioned incorrect estimated noise amount, the pixel value of interest P (x, y) is originally the smoothed model pixel value P ′ ( x, y) should not be replaced, but noise remains.

  On the other hand, FIG. 6 shows a case where the smoothing filter storing the sharpness of the edge according to the present embodiment is used, and a smoothing model more approximate to the steep edge of the original input image signal. A signal can be generated. That is, the smoothed model pixel value P ′ (x, y) in the vicinity of the steep edge is not far from the target pixel value P (x, y) as the input image signal, and even in the vicinity of such an edge. A certain amount of noise can be estimated. Further, even in the coring process in the vicinity of such a steep edge, the absolute difference value between the smoothed model pixel value P ′ (x, y) and the target pixel value P (x, y) that is the input image signal is obtained. The pixel value P (x, y) of interest is smoothed model pixel value P ′ (x) in the flat part near the edge, since it does not become larger than necessary and is compared with a probable estimated noise amount. , Y), and noise suppression can be performed up to the steep edge boundary.

  Next, an example in which the above-described noise reduction processing unit 100 is applied to a digital camera will be described with reference to FIGS. 9 to 11 show an example in which the noise reduction processing unit 100 is applied to reduce sensor noise of a digital camera that is an imaging system.

  First, FIG. 9 is a block diagram showing a configuration of a digital camera.

  As shown in FIG. 9, the digital camera includes an optical system 601, an imaging unit 602 that is an imaging unit, an image processing unit 603, a display unit 604, a recording unit 605, and an output unit 606. This is a configured imaging system.

  The optical system 601 includes a lens, an IR cut filter that is an optical filter, and an optical low-pass filter, and forms an image of passing light as an optical image on the image sensor of the imaging unit 602.

  The imaging unit 602 includes an imaging device, an electric signal amplifier, an A / D converter, and an imaging device controller. The imaging unit 602 photoelectrically converts an optical image formed on the imaging device by the optical system 601 to generate photoelectric signals. The converted signal of each pixel is amplified by an electric signal amplifier, the amplified analog signal is converted into a digital signal by an A / D converter, and further output to the image processing unit 603 by an image sensor controller.

  The image processing unit 603 performs noise reduction for each color of the image signal input from the imaging unit 602 and outputs the image signal to the display unit 604, the recording unit 605, and the output unit 606.

  The display unit 604 displays an image signal captured by the digital camera and input from the image processing unit 603 and other various information, and is configured as a liquid crystal monitor, for example.

  The recording unit 605 records the image information input from the image processing unit 603 on a recording medium.

  The output unit 606 outputs image information or the like input from the image processing unit 603 or the recording unit 605 to an external device or the like.

  Next, FIG. 10 is a block diagram showing in more detail the configuration of the image processing unit 603 in FIG. 9, and FIG. 11 is a block diagram showing in more detail the configuration of the noise reduction processing unit 100 in FIG.

  As illustrated in FIG. 10, the image processing unit 603 includes a color signal separation unit 701 and a noise reduction processing unit 100.

  The color signal separation unit 701 separates the image signal input from the imaging unit 602 into RGB color signals and outputs the separated RGB color signals to the noise reduction processing unit 100.

  As shown in FIG. 11, the noise reduction processing unit 100 includes a filtering calculation unit 101 and a coring calculation unit 102, and is the same as that shown in FIG. The noise reduction processing unit 100 performs noise suppression processing for each color signal separated by the color signal separation unit 701 as described above, and displays each color signal after noise reduction by a display unit 604, a recording unit 605, and an output unit. 606 and 606, respectively.

  According to the first embodiment, since the correlation between the target pixel and the surrounding pixels is detected and the smoothing filtering is performed by adaptively changing the filter coefficient based on the detected correlation, A smoothing model that preserves the structure and edges can be generated.

  And, by performing coring processing using a smoothing model that preserves the sharpness of edges and fine areas, noise near the edges and fine areas can be sufficiently reduced without smoothing the structure and edges of the fine areas. Is possible.

  Further, the pixel value deviation degree D (i, j) and the position deviation degree r (i, j) are calculated using a combination of relatively simple operations such as addition, subtraction, absolute value, square, and square root. Therefore, the processing load is reduced.

  In addition, the tuning value tune is added to the product of the pixel value divergence degree D (i, j) and the position divergence degree r (i, j), and the reciprocal thereof is obtained as the correlation index value Cor ( i, j), the filter coefficient of the surrounding pixel is calculated based on the correlation index value Cor (i, j), and the filter coefficient of the target pixel is calculated based on the pixel value deviation degree D (i, j). Therefore, it is possible to perform an operation by a relatively simple process such as a combination of multiplication, addition, and division.

  Further, in the calculation of the smoothing filtering coefficient, the filter coefficient may be calculated based only on the pixel value deviation degree D (i, j) without using the position deviation degree r (i, j). . Even in this case, it is possible to fulfill the function of generating the smoothed model signal in which the structure and edges of the fine area described above are stored, and it is possible to further simplify the arithmetic processing.

  Thus, since the processing load necessary for calculating the smoothing model is relatively small, it is possible to implement hardware for mounting in the imaging system.

[Embodiment 2]
12 and 13 show the second embodiment of the present invention, and FIG. 12 is a block diagram showing the configuration of the filtering operation unit. In the second embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals, description thereof is omitted, and only different points will be mainly described.

  The configuration of the noise reduction processing unit 100 of the present embodiment is the same as that shown in FIG. 1, and the internal configuration of the filtering operation unit 101 is slightly different from that of the first embodiment described above.

  That is, as shown in FIG. 12, the filtering calculation unit 101 includes a memory 201, a distance calculation unit 202, a difference calculation unit 203, a memory 204, a memory 205, a tuning constant storage unit 206, and a surrounding pixel coefficient calculation. A unit 207, a target pixel coefficient calculation unit 208, a memory 209, and a convolution operation unit 210, and further includes a region similarity determination unit 401 serving as a region similarity determination unit.

  The memory 201 is connected to the convolution operation unit 210 and further connected to the region similarity determination unit 401. The area similarity determination unit 401 is connected to the distance calculation unit 202, the difference calculation unit 203, and the memory 209.

  Next, the operation of the filtering operation unit 101 configured as shown in FIG. 12 will be described with reference to FIG. FIG. 13 is a flowchart showing the operation of the filtering operation unit 101.

  When pixels for a predetermined line that can be filtered are stored in the memory 201, for example, as shown in FIG. 4, the pixel of interest (pixel position (x, y)) and surrounding pixels (pixel position (x + i, y + j) )) Is extracted and output to the region similarity determination unit 401 and the convolution operation unit 210 (step S301).

  The region similarity determination unit 401 determines the pattern of pixels in the extracted attention region (step S302).

  Then, the region similarity determination unit 401 compares the pixel pattern in the attention region with the previously processed pixel pattern in the adjacent region to determine whether the similarity is high or low (step S303).

  When it is determined that the similarity is low, the region similarity determination unit 401 outputs each pixel of the attention region to the distance calculation unit 202 and the difference calculation unit 203. Thereby, similarly to the first embodiment described above, the calculation processing of the weighting factors Wcenter, W (i, j) as the filter coefficients is performed (step S304). Note that the filter coefficient calculation processing in step S304 is the same as the processing shown in steps S102 to S110 in FIG.

  The weighting factors Wcenter and W (i, j) calculated in this way are stored in the memory 209 (step S305).

  On the other hand, in step S303, when the region similarity determination unit 401 determines that the similarity between the pixel pattern in the attention region and the pixel pattern in the adjacent region processed previously is high, the process proceeds to steps S304 and S305. Without performing the process (that is, omitting the filter coefficient calculation process), the filter coefficient of the adjacent region that has been previously processed and is already stored in the memory 209 is used as it is for the following smoothing process.

  That is, when the process of step S305 is completed or when it is determined in step S303 that the similarity is high, the weighting factor Wcenter stored in the memory 209 is similar to the process shown in step S111 of FIG. And a convolution operation, which is a filter operation between W (i, j), the target pixel value P (x, y) and the peripheral pixel value P (x + i, y + j) in the region, as shown in Equation 10 above. Then, a smoothed model pixel value P ′ (x, y), which is a smoothed model signal, is calculated (step S306).

  Then, the convolution operation unit 210 outputs the calculated smoothed model pixel value P ′ (x, y) to the coring operation unit 102 (step S307).

  In addition, the comparison between the pixel pattern in the attention area currently set as described above and the pixel pattern in the adjacent area that has been processed before is not only performed on the area that has been previously processed. Alternatively, it may be performed for a plurality of areas processed in the past. However, in this case, it is necessary to store the past filter coefficients for the comparison target in the memory 209. Thereby, it is possible to increase the pattern-similar hit rate and further reduce the processing load.

  Also, the noise reduction processing unit 100 of the present embodiment can be applied to an imaging system such as a digital camera, as in the first embodiment.

  According to the second embodiment, the same effect as that of the first embodiment described above is obtained, and when the filter coefficient is calculated in the filtering calculation, the extracted pixel value pattern of the attention area is detected. If it is similar to the pattern of the region of interest for which the filter coefficient was calculated previously, the calculation of the new filter coefficient is omitted and the filter coefficient calculated previously is applied. It is possible to increase the processing speed.

[Embodiment 3]
14 and 15 show Embodiment 3 of the present invention, and FIG. 14 is a block diagram showing the configuration of the noise reduction processing unit. In the third embodiment, parts that are the same as those in the first and second embodiments are given the same reference numerals, description thereof is omitted, and only differences are mainly described.

  In the present embodiment, the tuning value for adjusting the degree of influence of the correlation-related value in the correlation index value Cor (i, j) can be variably controlled.

  That is, the noise reduction processing unit 100 of the present embodiment includes a filtering calculation unit 101 and a coring calculation unit 102, a frame memory 501 for storing an input image, and an input as an input SN calculation unit. An SN calculation unit 502, a frame memory 503 for storing an image after noise reduction, an SN improvement amount determination unit 504 serving as an SN calculation unit after noise reduction, and a tuning adjustment unit 505 serving as a tuning adjustment unit. It is configured.

  Here, the input original image is input to the filtering operation unit 101 and the coring operation unit 102, and is also input to the input SN calculation unit 502 via the frame memory 501. The input SN calculation unit 502 is connected to the SN improvement amount determination unit 504. The coring calculation unit 102 is connected to the SN improvement amount determination unit 504 via the frame memory 503. The SN improvement amount determination unit 504 is connected to the filtering operation unit 101 via the tuning adjustment unit 505. The SN improvement amount determination unit 504 outputs an output image.

  Next, the operation of the noise reduction processing unit 100 configured as shown in FIG. 14 will be described with reference to FIG. FIG. 15 is a flowchart showing the operation of the noise reduction processing unit 100.

  When this process is started, the input original image is stored in the frame memory 501 (step S401). The input SN calculation unit 502 extracts a flat area from the image signal of the input original image stored in the frame memory 501 (Step S402), and based on the variance of the pixel value of each pixel in the extracted flat area, the input SN value SNin is calculated, and the calculated SN value SNin is output to the SN improvement determination unit 504. The SN improvement amount determination unit 504 controls the tuning adjustment unit 505 based on the input SN value SNin, thereby causing the tuning constant storage unit 206 to set the initial value of the tuning value tune (step S403).

  Further, the filtering operation unit 101 performs smoothing by performing a filtering operation on the input image signals (P (x, y) and P (x + i, y + j)) in the region of interest in the same manner as the processing shown in FIG. And outputs the smoothed model pixel value P ′ (x, y), which is a smoothed model signal, to the coring operation unit 102 (step S404).

  Similar to the processing shown in FIG. 8, the coring calculation unit 102 receives the target pixel value P (x, y) that is the input image signal and the smoothed model pixel value P ′ (x, y) that is the smoothed model signal. ) To suppress the noise and calculate an output signal Pout that is a candidate for the final output signal Pout (step S405).

The output signal Pout of the coring calculation unit 102 is stored in the frame memory 503 (step S406), and the SN improvement amount determination unit 504 calculates the SN value SNin of the input image signal from the output signal Pout of the coring calculation unit 102. The same area as the flat area is extracted (step S407), and the output SN value SNout is calculated based on the variance of the pixel values of each pixel in this area. Then, the SN improvement amount determination unit 504 calculates the difference between the output SN value SNout and the input SN value SNin input from the input SN calculation unit 502 as shown in the following Expression 12, and the SN improvement amount ΔSN (Step S408).
[Equation 12]

  Next, the SN improvement amount determination unit 504 calculates the difference between the SN improvement amount ΔSN calculated in step S408 and the preset target SN improvement amount ΔSNgoal (step S409), and the absolute value of the calculated difference Is less than or equal to a certain threshold Th (step S410).

  When it is determined that the absolute value of the difference exceeds a certain threshold Th, the SN improvement amount determination unit 504 outputs the SN improvement amount ΔSN and the target SN improvement amount ΔSNgoal to the tuning adjustment unit 505. Then, the tuning adjustment unit 505 first determines whether or not the SN improvement amount ΔSN is smaller than the target SN improvement amount ΔSNgoal (step S411).

  Here, if the tuning adjustment unit 505 determines that the SN improvement amount ΔSN is smaller than the target SN improvement amount ΔSNgoal, the tuning adjustment unit 505 increases the tuning value tune used for the filtering calculation (step S412). When the tuning value tune is increased, the degree of influence of the correlation-related value in the correlation index value Cor (i, j) decreases, so that the filter coefficient approaches more uniformly and the smoothing effect becomes stronger, so that the edge or fine region is increased. The effect of preserving sharpness is weakened.

  When the tuning adjustment unit 505 determines that the SN improvement amount ΔSN is larger than the target SN improvement amount ΔSNgoal, the tuning adjustment unit 505 decreases the tuning value tune used for the filtering calculation (step S413). When the tuning value tune is decreased, the degree of influence of the correlation value on the correlation index value Cor (i, j) increases, so that weighting that greatly depends on the correlation value is performed, and the smoothing effect is weak. As a result, the effect of preserving the sharpness of the edges and fine regions becomes stronger.

  Then, the tuning adjustment unit 505 sets the tuning value “tune” calculated by the processing in step S412 or step S413 as a new tuning value in the tuning constant storage unit 206 (step S414). Thereafter, the process returns to step S404, and the noise reduction process as described above is repeated.

  By performing such processing, if it is determined in step S410 that the absolute value of the difference between the SN improvement amount ΔSN and the target SN improvement amount ΔSNgoal is equal to or less than a certain threshold Th, the SN improvement amount determination unit 504 Outputs the output signal Pout from the coring operation unit 102 as it is as the final output signal Pout (step S415).

  Note that the noise reduction processing unit 100 of the present embodiment can also be applied to an imaging system such as a digital camera, as in the first and second embodiments.

  According to the third embodiment, the effects similar to those of the first and second embodiments described above are obtained, and the tuning value is adaptively changed according to the image. The balance between the effect of preserving the sharpness of the fine region and the effect can be controlled.

  Since the noise reduction process is automatically repeated while changing the tuning value until a preset target SN improvement amount is achieved, an output image tuned to the optimum smoothing intensity can be obtained. .

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the spirit of the invention.

  The present invention can be suitably used for an imaging system, an image processing program, and an image processing method for calculating a smoothing model used for image noise reduction.

The block diagram which shows the structure of the noise reduction process part in Embodiment 1 of this invention. The block diagram which shows the structure of the filtering calculating part in the said Embodiment 1. FIG. The block diagram which shows the structure of the coring calculating part in the said Embodiment 1. FIG. FIG. 3 is a diagram illustrating an example of a relationship between a pixel of interest in a region of interest and each peripheral pixel in the first embodiment. FIG. 5 is a diagram showing a manner in which the relationship between the weighting coefficient of the target pixel and the weighting coefficients of surrounding pixels changes depending on the sum of the pixel value divergence degrees in the first embodiment. In the said Embodiment 1, the figure which shows the mode of the noise suppression of the edge part by a coring process. The flowchart which shows the effect | action of the filtering calculating part in the said Embodiment 1. FIG. The flowchart which shows the effect | action of the coring process part in the said Embodiment 1. FIG. FIG. 2 is a block diagram illustrating a configuration of a digital camera according to the first embodiment. The block diagram which shows the structure of the image process part in FIG. 9 of the said Embodiment 1 in detail. The block diagram which shows the structure of the noise reduction process part in FIG. 10 of the said Embodiment 1 in detail. The block diagram which shows the structure of the filtering calculating part in Embodiment 2 of this invention. The flowchart which shows the effect | action of the filtering calculating part in the said Embodiment 2. The block diagram which shows the structure of the noise reduction process part in Embodiment 3 of this invention. 10 is a flowchart showing the operation of the noise reduction processing unit in the third embodiment. The figure which shows notionally the correlation of the weighting coefficient of the attention pixel and surrounding pixel in the conventional filtering process. The figure which shows notionally the mode that the noise of an edge part remains in the conventional coring process.

Explanation of symbols

100: Noise reduction processing unit (at least a part of the imaging system)
101... Filtering operation unit (smoothing signal calculation means, attention area setting means)
102: Coring calculation unit (coring calculation means)
201, 204, 205, 209 ... Memory 202 ... Distance calculation section (smoothing signal calculation means, position deviation degree calculation means)
203... Difference calculation unit (smoothing signal calculation means, pixel value deviation degree calculation means)
206 ... Tuning constant storage unit 207 ... Peripheral pixel coefficient calculation unit (smoothing signal calculation means, correlation related value calculation means, correlation index value calculation means, filter coefficient calculation means)
208... Pixel pixel calculation unit (smoothing signal calculation means, filter coefficient calculation means)
210: Convolution calculation unit (smoothing signal calculation means, filter calculation means)
301 ... Noise model LUT (noise amount estimating means)
302 ... ISO sensitivity storage unit (noise amount estimation means)
303 ... Gain control section (noise amount estimating means)
304: Noise amount correction unit (noise reduction means)
305 ... Coring processing unit (noise reduction means)
401... Area similarity determination unit (area similarity determination means)
501... Frame memory 502... Input SN calculation unit (input SN calculation means)
503... Frame memory 504... SN improvement amount determination unit (SN calculation means after noise reduction)
505 ... Tuning adjustment unit (tuning adjustment means)
601: Optical system 602 ... Imaging unit (imaging means)
603 ... Image processing unit 604 ... Display unit 605 ... Recording unit 606 ... Output unit 701 ... Color signal separation unit

Claims (13)

In an imaging system that performs noise reduction processing on a video signal captured from an imaging system,
Based on at least one of spatial correlation and pixel value correlation between a target pixel to be subjected to noise reduction processing in the video signal and one or more peripheral pixels located in the spatial vicinity of the target pixel. Smoothing signal calculation means for calculating a smoothing signal for the pixel of interest based on the correlation-related value calculated in the above and an adjustment value for adjusting the correlation-related value;
Noise amount estimating means for estimating the amount of noise included in the target pixel based on the smoothed signal of the target pixel calculated by the smoothed signal calculating means;
Noise reduction means for performing noise reduction processing on the pixel of interest based on the smoothed signal calculated by the smoothed signal calculation means and the noise amount estimated by the noise amount estimation means;
An imaging system comprising: The smoothed signal calculation means includes:
Attention area setting means for setting an attention area constituted by a target pixel and one or more peripheral pixels located in the spatial vicinity of the target pixel in an input image signal;
Pixel value divergence calculating means for calculating a pixel value divergence degree for each peripheral pixel indicating a degree of divergence between the pixel of interest and the peripheral pixels;
A position divergence calculating means for calculating a position divergence degree indicating the degree of spatial divergence between the target pixel and the peripheral pixels for each peripheral pixel;
By multiplying the pixel value divergence degree and the position divergence degree related to the same peripheral pixel for each peripheral pixel, a correlation related value, which is a value related to the correlation between the target pixel and the peripheral pixel, is obtained. Correlation-related value calculating means for each peripheral pixel;
A tuning value that is a constant for adjustment is added to the correlation-related value, and based on the added value, a correlation index value that serves as an index indicating the correlation between the target pixel and the peripheral pixel is determined for each peripheral pixel. A correlation index value calculating means to be obtained;
A filter coefficient computing means for obtaining, for each pixel, filter coefficients for all pixels included in the region of interest based on at least one of the correlation index value and a value obtained in the process of obtaining the correlation index value;
2. The imaging system according to claim 1, further comprising: a filter calculation unit that calculates a smoothed model pixel value related to the target pixel by performing a filter operation on the target region using the filter coefficient. .   The filter coefficient calculation means increases the filter coefficient for the target pixel relative to the filter coefficient for the peripheral pixel as the sum of the pixel value divergence degrees for all the peripheral pixels in the target area increases. 3. The imaging system according to claim 2, wherein a filter coefficient for the target pixel and a filter coefficient for the peripheral pixel are obtained.   The filter coefficient calculation means has a filter coefficient for the pixel of interest that is not less than a predetermined upper limit value of 0 or more and 1 or less as the sum of the pixel value divergence degrees for all the surrounding pixels in the region of interest increases. By increasing the range, and subtracting the filter coefficient for the target pixel from 1 as a factor of the filter coefficient for the peripheral pixel, the filter coefficient for the target pixel is relative to the filter coefficient for the peripheral pixel. The imaging system according to claim 3, wherein the imaging system is configured to be large. The position of the pixel of interest is (x, y) (where x and y are real numbers), and the position of the peripheral pixel is (x + i, y + j) (where i and j are real numbers, i and j Are not simultaneously 0.) When the pixel value of the pixel of interest is P (x, y) and the pixel value of the peripheral pixels is P (x + i, y + j),
The pixel value divergence degree calculation means calculates the pixel value divergence degree D (i, j) as follows:

Or the following formula

Is calculated by
The position divergence calculating means calculates the position divergence r (i, j) by the following formula.

Or the following formula

Is calculated by
The correlation related value calculating means calculates the correlation related value by

Is calculated by
The correlation index value calculation means calculates the correlation index value Cor (i, j) based on the correlation-related value and the tuning value tune (where the tuning value tune is a real number greater than or equal to 0) as follows:

Is calculated by
The filter coefficient calculation means calculates the filter coefficient Wcenter of the target pixel by the following formula

(Here, the sum is taken for (i, j) corresponding to all peripheral pixels included in the region of interest (that is, excluded from the sum if i = 0 and j = 0)) And the filter coefficient W (i, j) of the surrounding pixels is expressed by the following formula:

(Here, the sum is taken for (i, j) corresponding to all peripheral pixels included in the region of interest (that is, excluded from the sum if i = 0 and j = 0)) And)
The filter calculation means calculates the smoothed model pixel value P ′ (x, y) related to the target pixel by the following formula:

(Here, the sum is taken for (i, j) corresponding to all peripheral pixels included in the region of interest (that is, excluded from the sum if i = 0 and j = 0)) The imaging system according to claim 3, wherein The region of interest setting means sequentially sets the region of interest so as to sequentially change the pixel position of the pixel of interest.
A region similarity determination unit that determines whether the attention region currently set by the attention region setting unit and the attention region set in the past by the attention region setting unit are similar to each other;
If it is determined by the region similarity determining means that the images are similar, the pixel value divergence calculating means, the position divergence calculating means, the correlation related value calculating means, the correlation index value calculating means, and the filter coefficient calculating means, In which the filter calculation means calculates the smoothed model pixel value using the filter coefficients already obtained for the attention area set in the past for the attention area currently set. The imaging system according to claim 2, wherein:   The apparatus further comprises coring calculation means for performing a coring process using the pixel value of the target pixel related to the input image signal and the smoothed model pixel value related to the target pixel. Item 3. The imaging system according to Item 2.   The imaging system according to claim 7, further comprising tuning adjustment means for changing the tuning value in accordance with the input image signal and the image signal after the coring process. Input SN calculation means for calculating the SN ratio of the input image signal;
A noise-reduced SN calculation means for calculating an SN ratio of the image signal after the coring process;
Further comprising
The tuning adjustment unit is configured to change the tuning value in accordance with an S / N ratio of the input image signal and an S / N ratio of the image signal after the coring process. 9. The imaging system according to 8. The input SN calculation means extracts a flat area of the input image signal and calculates an SN ratio of the flat area.
The imaging system according to claim 9, wherein the noise-reduced SN calculation means calculates an SN ratio of the same area as the flat area in the image signal after the coring process. It further has an imaging means for capturing an image and outputting an image signal,
The imaging system according to claim 2, wherein the image signal output from the imaging means is the input image signal. On the computer,
An attention area setting step for setting an attention area composed of the attention pixel and one or more peripheral pixels located in the spatial vicinity of the attention pixel in the input image signal;
A pixel value divergence calculating step for calculating a pixel value divergence degree indicating the degree of divergence between the pixel of interest and the peripheral pixel for each peripheral pixel;
A position divergence degree calculating step for calculating a position divergence degree indicating the degree of spatial divergence between the target pixel and the peripheral pixels for each peripheral pixel;
By multiplying the pixel value divergence degree and the position divergence degree related to the same peripheral pixel for each peripheral pixel, a correlation related value, which is a value related to the correlation between the target pixel and the peripheral pixel, is obtained. Correlation-related value calculation step for each peripheral pixel;
A tuning value that is a constant for adjustment is added to the correlation-related value, and based on the added value, a correlation index value that serves as an index indicating the correlation between the target pixel and the peripheral pixel is determined for each peripheral pixel. A correlation index value calculation step to be obtained;
A filter coefficient calculation step for obtaining, for each pixel, filter coefficients for all pixels included in the region of interest based on at least one of the correlation index value and a value obtained in the process of obtaining the correlation index value;
A filter calculation step of calculating a smoothed model pixel value related to the target pixel by performing a filter operation on the target region using the filter coefficient;
An image processing program for executing An attention area setting step for setting an attention area composed of the attention pixel and one or more peripheral pixels located in the spatial vicinity of the attention pixel in the input image signal;
A pixel value divergence calculating step for calculating a pixel value divergence degree indicating the degree of divergence between the pixel of interest and the peripheral pixel for each peripheral pixel;
A position divergence degree calculating step for calculating a position divergence degree indicating the degree of spatial divergence between the target pixel and the peripheral pixels for each peripheral pixel;
By multiplying the pixel value divergence degree and the position divergence degree related to the same peripheral pixel for each peripheral pixel, a correlation related value, which is a value related to the correlation between the target pixel and the peripheral pixel, is obtained. Correlation-related value calculation step for each peripheral pixel;
A tuning value that is a constant for adjustment is added to the correlation-related value, and based on the added value, a correlation index value that serves as an index indicating the correlation between the target pixel and the peripheral pixel is determined for each peripheral pixel. A correlation index value calculation step to be obtained;
A filter coefficient calculation step for obtaining, for each pixel, filter coefficients for all pixels included in the region of interest based on at least one of the correlation index value and a value obtained in the process of obtaining the correlation index value;
A filter calculation step of calculating a smoothed model pixel value related to the target pixel by performing a filter operation on the target region using the filter coefficient;
An image processing method comprising:

Images