CN100550996C - Image processing apparatus, imaging device and image processing method - Google Patents

Abstract

Image information acquisition unit (101) obtains image information.Picture content separative element (102) is separated into monochrome information and color information with the image information that image information acquisition unit (101) obtain.Edge extracting unit (110) extracts marginal information from the monochrome information that picture content separative element (102) separates.Brightness noise is removed unit (105) and is removed noise from the monochrome information that picture content separative element (102) separates.Color noise removing unit (106) is removed noise from the color information that picture content separative element (102) separates.The marginal information that image information assembled unit (107) extracts based on edge extracting unit (110) is removed the color information composograph information that unit (105) has been removed the monochrome information of noise and removed noise by color noise removing unit (106) by brightness noise.

Description

Image processing device, imaging device and image processing method

technical field

The invention relates to an image processing device, an imaging device and an image processing method.

Background technique

This application incorporates by reference the entire contents of Japanese Priority Application No. 2006-208063 filed in Japan on July 31, 2006 and Japanese Priority Application No. 2007-034532 filed in Japan on February 15, 2007.

In recent years, in the field of digital still cameras (hereinafter referred to as "digital cameras"), an increase in the number of pixels of charge-coupled devices (CCDs) or imaging devices has been achieved. On the other hand, such an increase in the number of CCD pixels causes a problem of a decrease in the sensitivity of the CCD.

In order to solve such a problem, an imaging device is disclosed in which pixels of a plurality of images are added (see Japanese Patent Application Laid-Open No. 2005-44915). In such an imaging device, pixels of an image are increased to increase light sensitivity.

Also, another imaging device is disclosed in which a pixel signal for adding together pixel values of adjacent pixels is output to increase sensitivity (see Japanese Patent Application Laid-Open No. 2005-303519).

In addition, there is also disclosed a technique in which although noise is enhanced when an image is captured with increased sensitivity, the cutoff frequency of the low-pass filter is set according to the imaging sensitivity to remove noise (see Japanese Patent Application Laid-Open No. 2004-297731 ).

However, in the technique disclosed in Japanese Patent Application Laid-Open No. 2005-44915, in the case of increasing the pixels of an image, the exposure time is increased. No problem with the camera fixed and the target not moving. However, if either the camera or the target moves, a positional deviation will disadvantageously occur.

Also, in the technique disclosed in Japanese Patent Application Laid-Open No. 2005-303519, pixel values of adjacent pixels are added together, thereby causing a problem of lowering resolution.

Furthermore, in the technique disclosed in Japanese Patent Application Laid-Open No. 2004-297731, although noise can be removed according to the imaging sensitivity, the edges of the image become blurred. For example, if you image at a location lit by a light with the ISO set to high, blurring will be noticeable to the image even if there is less noise. It also has a noticeable effect on the image, causing unnecessary blurring of the image.

Furthermore, in the technique disclosed in Japanese Patent Application Laid-Open No. 2004-297731, when the exposure time is short in imaging, color reproducibility and white balance deteriorate, and the color and brightness of the image are not improved even if noise is removed balance between.

Contents of the invention

It is an object of the present invention to at least partly solve the above-mentioned problems of the prior art.

An image processing device according to one aspect of the present invention includes: an image information acquisition unit for acquiring image information; an image component separation unit that separates the image information acquired by the image information acquisition unit into brightness information and color information; an edge extraction unit that extracts the image information from the image components the luminance information separated by the separation unit extracts edge information; the luminance noise removal unit removes noise from the luminance information separated by the image component separation unit; the color noise removal unit removes noise from the color information separated by the image component separation unit; and the image information synthesis unit, Based on the edge information extracted by the edge extracting unit, the luminance information from which noise has been removed by the luminance noise removing unit, and the color information from which noise has been removed by the color noise removing unit synthesize image information.

An image processing device according to another aspect of the present invention includes: an image information acquisition unit for acquiring image information; a high-sensitivity low-resolution image generation unit that compares a plurality of adjacent pixel values in the image information acquired by the image information acquisition unit A pixel value is added to generate a high-sensitivity low-resolution image; the scaling unit enlarges or reduces the high-sensitivity low-resolution image information generated by the high-sensitivity low-resolution image generation unit; and the image information combination unit, from the image information The image information acquired by the acquisition unit and the high-sensitivity low-resolution image information enlarged or reduced by the scaling unit are used to synthesize the image information.

An image processing method according to another aspect of the present invention, comprising: acquiring image information; separating the image information acquired in the acquiring step into luminance information and color information; extracting edge information from the luminance information separated in the separating step; and removing luminance noise, including noise removal from the luminance information separated in the separation step; color noise removal comprising removing noise from the color information separated in the separation step; and based on the edge information extracted in the extraction step, the luminance information from which the noise was removed in the luminance noise removal step, and The color information from which noise has been removed in the color noise removal step generates image information.

An image processing method according to another aspect of the present invention, comprising: acquiring image information; separating the image information acquired in the acquiring step into brightness information and color information; extracting edge information from the brightness information separated in the separating step; The luminance information and color information of the scaling step are scaled; the luminance noise removal includes removing noise from the luminance information scaled in the scaling step; the color noise removal includes removing noise from the color information scaled in the scaling step, based on the edges extracted in the extraction step information, the luminance information from which noise has been removed in the luminance noise removal step, and the color information from which noise has been removed in the color noise removal step to generate image information.

The image processing method according to another aspect of the present invention includes: obtaining image information; generating a high-sensitivity low-resolution image by adding a plurality of adjacent pixel values in the image information obtained in the obtaining step to generate a pixel value; enlarging or reducing the high-sensitivity and low-resolution image information generated in the generating step; and synthesizing the image information from the image information obtained in the obtaining step and the enlarged or reduced high-sensitivity and low-resolution image information in the scaling step.

The above and other objects, features, advantages and technical and industrial significance of this invention will be better understood by reading the following detailed description of presently preferred embodiments of the invention, when considered in conjunction with the accompanying drawings.

Description of drawings

1 is a block diagram of the structure of an image processing unit according to a first embodiment of the present invention;

2 is a table for explaining an example of a data structure of an edge extraction filter size database;

FIG. 3 is a table for illustrating an example of a data structure of a parameter database;

4 is a table for illustrating an example of a data structure of a noise removal filter size database;

5 is a table for illustrating an example of a data structure of a Gaussian sigma value database;

Fig. 6A is an image information acquisition unit, a component separation unit, an imaging condition acquisition unit, a filter determination unit, a brightness component edge extraction unit, a brightness component noise removal unit, a color component noise removal unit, an image information synthesis unit, and an image information compression unit, and a flowchart of the image processing process performed by the image information output unit;

Fig. 6B is an image information acquisition unit, a component separation unit, an imaging condition acquisition unit, a filter determination unit, a brightness component edge extraction unit, a brightness component noise removal unit, a color component noise removal unit, an image information synthesis unit, and an image information compression unit, and a flowchart of the image processing process performed by the image information output unit;

7 is a diagram for explaining an example of an edge extraction filter having a filter size of 5×5;

FIG. 8 is a diagram for explaining an example of a result of edge extraction using an edge extraction filter;

FIG. 9 is a diagram for explaining the result of noise removal using a luma filter;

FIG. 10 is a diagram for explaining a result obtained by combining edge information and noise-removed luminance information;

11 is a block diagram of a hardware configuration of a digital camera according to the first embodiment;

12 is a block diagram of a structure of an image processing unit according to a second embodiment of the present invention;

13 is a table for explaining an example of a data structure of a scaling factor database according to a second embodiment;

Figure 14A is an image information acquisition unit, component separation unit, imaging condition acquisition unit, filter determination unit, scaling unit, brightness component edge extraction unit, brightness component noise removal unit, color component noise removal unit, reverse scaling unit, image information A flowchart of the image processing process performed by the synthesis unit, the image information compression unit, and the image information output unit;

Figure 14B is an image information acquisition unit, component separation unit, imaging condition acquisition unit, filter determination unit, scaling unit, brightness component edge extraction unit, brightness component noise removal unit, color component noise removal unit, reverse scaling unit, image information A flowchart of the image processing process performed by the synthesis unit, the image information compression unit, and the image information output unit;

15 is a block diagram of the structure of an image processing apparatus according to a third embodiment of the present invention;

16A is an image information acquisition unit, an image information conversion unit, a component separation unit, an imaging condition acquisition unit, a filter determination unit, a brightness component edge extraction unit, a brightness component noise removal unit, a color component noise removal unit, and an image information synthesis unit, A flowchart of the image processing process performed by the image information conversion unit and the image information output unit;

16B is an image information acquisition unit, an image information conversion unit, a component separation unit, an imaging condition acquisition unit, a filter determination unit, a brightness component edge extraction unit, a brightness component noise removal unit, a color component noise removal unit, and an image information synthesis unit, A flowchart of the image processing process performed by the image information conversion unit and the image information output unit;

17 is a block diagram of a hardware configuration of an image processing apparatus according to a third embodiment;

18 is a block diagram of an image processing unit according to a fourth embodiment of the present invention;

Figure 19 is an image information acquisition unit, a high-sensitivity low-resolution image generation unit, a component separation unit, a brightness component edge extraction unit, a brightness component noise removal unit, a color component noise removal unit, a scaling unit, a brightness component synthesis unit, and image information A flowchart of the image processing process performed by the synthesis unit and the image information output unit;

FIG. 20 is a diagram for explaining an example of a smoothing filter having a filter size of 3×3;

FIG. 21 is a diagram for explaining an example of an edge extraction filter having a filter size of 5×5;

FIG. 22 is a diagram for explaining separated luminance information;

FIG. 23 is a diagram for explaining results obtained by extracting edges from luminance information;

FIG. 24 is a diagram illustrating a result obtained by combining luminance information and edge information;

25 is a block diagram of the configuration of an image processing apparatus according to a fifth embodiment of the present invention;

26 is an image information acquisition unit, component conversion unit, component separation unit, color component noise removal unit, scaling unit, brightness component noise removal unit, brightness component edge extraction unit, brightness component synthesis unit, image information synthesis unit, and image information Flowchart of the image processing process performed by the output unit.

Detailed ways

Exemplary embodiments of the present invention are described in detail below with reference to the accompanying drawings.

Referring to the drawings, a first embodiment of the present invention will be described. First, a structural example of an image processing unit included in a digital camera to which the present invention is applied is explained. FIG. 1 is a block diagram of the structure of an image processing unit 100 according to the first embodiment.

The image processing unit 100 includes an image information acquisition unit 101, a component separation unit 102, an imaging condition acquisition unit 103, a filter determination unit 104, a brightness component edge extraction unit 110, a brightness component noise removal unit 105, a color component noise removal unit 106, an image Information synthesis unit 107 , image information compression unit 108 , image information output unit 109 , edge extraction filter size database 120 , parameter database 130 , noise removal filter size database 140 , and Gaussian σ value database 150 .

The edge extraction filter size is the size of a filter used in extracting edges from image information. The optimum size of a filter used in edge extraction varies depending on imaging conditions. The edge extraction filter size database 120 stores an optimum filter size for each imaging condition. FIG. 2 is a table for explaining an example of a data structure of an edge extraction filter size database. In this way, in the edge extraction filter size database 120, imaging conditions and edge extraction filter sizes are associated with each other. Therefore, an optimal filter size can be determined from imaging conditions.

Imaging conditions are conditions that affect edge extraction in images captured by a digital camera. In particular, the sensitivity of the camera, the exposure time, and the temperature at the time of shooting are defined as imaging conditions. The light sensitivity of a camera refers to that of a CCD or Complementary Metal Oxide Semiconductor (CMOS) sensor. When the sensitivity is high, the shutter speed is high, camera shake tends not to occur, and moving subjects can be shot without blurring. In addition, bright images can be captured even in dark places. On the other hand, if the sensitivity is increased, noise tends to be generated.

Exposure time refers to the time when light hits the CCD or CMOS sensor. If the sensitivity is high, images can be captured even if the exposure time is reduced. The temperature at the time of shooting refers to the outside air temperature at the time of shooting with the camera. When the temperature is low, it is not easy to generate noise.

According to the first embodiment, the sensitivity of the camera, the exposure time, and the temperature at the time of shooting are used as imaging conditions. However, it does not mean that the imaging conditions are limited to those described above, but any imaging conditions that change edge extraction may be used.

The parameter database 130 stores σ values and k values used to calculate values of Laplacian of Gaussian (LoG) filters used for edge extraction corresponding to imaging conditions. FIG. 3 is a table for explaining an example of a data structure of a parameter database. The parameter database 130 stores the imaging conditions and the σ value and the k value used to calculate the value of the LoG filter in association with each other.

The σ value refers to a parameter that determines the width of the filter. When the value of σ is larger, the filter width is wider and the smoothing effect is larger. When the σ value is small, the filter width is narrow and the effect of edge enhancement is strong.

The k value refers to a parameter representing the strength of edge enhancement. When the value of k is larger, the effect of edge enhancement is larger. When the value of k is small, the effect of blur restoration is small. In this way, when changing the σ value and the k value, the correction result can be adjusted. Here, when the σ value of the LoG filter is closer to zero, the filter is closer to the Laplacian filter.

Not only edge extraction filters are found from the LoG function, filters found from other functions can be used to perform the edge extraction process.

The noise removal filter size database 140 stores filter sizes for noise removal corresponding to imaging conditions. FIG. 4 is a table for explaining an example of the data structure of the noise removal filter size database. The noise removal filter size database 140 stores imaging conditions and filter sizes in association with each other.

The filter size refers to the size of a noise removal filter specified by imaging conditions, and the filter size is stored for each luminance information and color information. When the filter size is larger, the effect of noise removal is larger, but the edge blurring is larger. That is, the noise removal effect and edge blurring have such a trade-off relationship. Since the noise level of the captured image varies according to the imaging conditions, the filter size is selected according to the imaging conditions at the time of shooting to perform optimal noise removal.

The human eye is sensitive to brightness changes but insensitive to color changes. Therefore, the filter size of luminance information is set to be smaller than that of color information, thereby performing effective noise removal in consideration of the characteristics of human eyes. Image information in the YUV format consists of brightness information (Y) and color information (U, V). The luminance information refers to a value approximately proportional to the intensity perceived as "brightness" by human eyes. In the color information (U, V), U represents the hue and chroma of the blue system, and V represents the hue and chroma of the red system. Corresponding noise removal processing is performed for each luminance information and color information, thereby optimally removing noise. In this embodiment, in the noise removal filter size database 140, the first size information indicating the size of the luma filter and the second size information indicating the size of the color filter store the same size. Here, the size of the brightness filter indicated by the first size information may be smaller than the size of the color filter indicated by the second size information.

The Gaussian σ value database 150 stores σ values used to calculate values of Gaussian smoothing filters for noise removal corresponding to imaging conditions. FIG. 5 is a table for explaining an example of the data structure of the Gaussian σ value database. The Gaussian σ value database 150 stores imaging conditions and σ values used to calculate the value of the Gaussian smoothing filter in association with each other.

The magnitude of the σ value represents the strength of noise removal. When the σ value is larger, the noise removal effect is larger. Here, the filter is not only found from the Gaussian function. Alternatively, filters found from other functions can be used to perform the noise removal process.

The image information acquisition unit 101 acquires image information from a temporary storage memory. The obtained image information is image information converted into YUV format. The component separation unit 102 separates the image information in YUV format acquired by the image information acquisition unit 101 into brightness information (Y) and color information (U, V).

The imaging condition acquisition unit 103 acquires imaging conditions corresponding to the image information acquired by the image information acquisition unit 101 from the temporary storage memory. The imaging conditions are the imaging conditions at the time of shooting, that is, the sensitivity of the camera, the exposure time, and the temperature at the time of shooting, and they are stored in association with each other. Here, imaging conditions at the time of shooting may be stored as part of the image information.

The filter determination unit 104 determines an edge extraction filter and a noise removal filter (brightness filter and color filter) corresponding to imaging conditions. First, the filter determination unit 104 specifies the size of the edge extraction filter associated with the imaging condition acquired by the imaging condition acquisition unit 103 from the edge extraction filter size database 120, and specifies the size of the edge extraction filter associated with the imaging condition acquisition unit 103 from the parameter database 130. The σ and k values of the LoG function associated with the imaging conditions.

The filter determination unit 104 further calculates an edge extraction filter by using the following equation (1) using the edge extraction filter size and the σ value of the LoG function.

$\mathrm{<span\; class="notranslate">\; Log</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; \&pi;\&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="2"\; label="y"\; filenames="C20071013975800381.png"\; state="\{\{state\}\}">y</figure-callout></span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; ]</span>{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="2"\; label="y"\; filenames="C20071013975800381.png"\; state="\{\{state\}\}">y</figure-callout></span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

And, the filter determination unit 104 determines a noise removal filter corresponding to the imaging condition. The filter determination unit 104 specifies the size of the noise removal filter associated with the imaging condition acquired by the imaging condition acquisition unit 103 from the noise removal filter size database 140, that is, the size and The size of the color filter used to remove noise from color information. In addition, the filter determination unit 104 specifies the σ value of the Gaussian function associated with the imaging condition acquired by the imaging condition acquisition unit 103 from the Gaussian σ value database 150 .

The filter determination unit 104 further calculates a luminance filter (Gaussian smoothing filter) by using the following equation (2) using the size of the luminance filter and the σ value of the Gaussian function.

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="2"\; label="y"\; filenames="C20071013975800381.png"\; state="\{\{state\}\}">y</figure-callout></span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Also, the filter determination unit 104 calculates a color filter (Gaussian smoothing filter) using the above-described equation (2) using the size of the color filter and the σ value of the Gaussian function.

The luminance component edge extraction unit 110 extracts edge information from the luminance information using the edge extraction filter determined by the filter determination unit 104 and the k value. The edge information extraction result is expressed by the following equation (3).

$\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; iGO</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CircleTimes;</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

where the symbols of the operations in Equation (3) represent convolution processing.

The luminance component noise removal unit 105 removes noise from luminance information using the noise removal filter determined by the filter determination unit 104 , that is, a luminance filter (low-pass filter). The color component noise removal unit 106 removes noise from color information using the noise removal filter determined by the filter determination unit 104 , that is, a color filter (low-pass filter).

Image information synthesis unit 107 combines edge information extracted from luminance component edge extraction unit 110, luminance information from which noise has been removed by luminance component noise removal unit 105, and color information from which noise has been removed by color component noise removal unit 106 to generate YUV format image information. The image information synthesis unit 107 forms an image information generation unit according to the present invention. The combination of edge information and luminance information is calculated by the following equation (4).

$\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; iGO</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CircleTimes;</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Where fs(x, y) represents the luminance component processed by the noise removal filter, and s(x, y) represents the synthesized image. The luminance information calculated by equation (4) is further combined with color information to generate image information. Here, the image information in YUV format can be further converted into image information in other formats, such as RGB format.

The image information compression unit 108 compresses the image information in the YUV format synthesized by the image information synthesis unit 107 into, for example, a Joint Photographic Experts Group (JPEG) format. The image information output unit 109 outputs the image information compressed by the image information compression unit 108 to a memory card or the like.

Next, image processing by the image processing unit 100 configured as described above will be described. 6A and 6B are image information acquisition unit, component separation unit, imaging condition acquisition unit, filter determination unit, brightness component edge extraction unit, brightness component noise removal unit, color component noise removal unit, image information synthesis unit, image information compression unit, and a flowchart of the image processing process performed by the image information output unit.

The image information acquiring unit 101 acquires image information in YUV format from a temporary storage memory (step S601). The component separation unit 102 separates the image information in YUV format acquired by the image information acquisition unit 101 into brightness information and color information (step S602). The imaging condition acquisition unit 103 acquires imaging conditions associated with image information (step S603).

The filter determination unit 104 specifies the size of the edge extraction filter corresponding to the imaging condition acquired by the imaging condition acquisition unit 103 from the edge extraction filter size database 120 (step S604 ). For example, specify 5×5 as the size of the edge extraction filter.

The filter determination unit 104 specifies the σ value and the k value of the LoG function corresponding to the imaging condition acquired by the imaging condition acquisition unit 103 from the parameter database 130 (step S605 ). The filter determination unit 104 determines each value of the edge extraction filter from the filter size of the edge extraction filter and the σ value and the k value of the LoG function (step S606 ). For example, FIG. 7 is a diagram illustrating an example of an edge extraction filter having a filter size of 5×5. Ai, which is a value of the edge extraction filter, is a value calculated by equation (1).

The luminance component edge extraction unit 110 extracts edge information from the luminance information using the edge extraction filter determined by the filter determination unit 104 (step S607). FIG. 8 is a diagram for explaining an example of a result of edge extraction using an edge extraction filter. As shown in FIG. 8, edge components (edge information) are extracted from image information.

The filter determination unit 104 specifies the size of the filter corresponding to the imaging condition acquired by the imaging condition acquisition unit 103 from the noise removal filter size database 140 (step S608 ). Specifically, the size of the luminance filter and the size of the color filter corresponding to the sensitivity of the camera as imaging conditions, the exposure time, and the temperature at the time of shooting are specified.

The filter determination unit 104 determines the σ value of the Gaussian function corresponding to the imaging condition acquired by the imaging condition acquisition unit 103 from the Gaussian σ value database 150 (step S609 ).

The filter determination unit 104 determines a luminance filter from the filter size of the luminance filter and the specified σ value of the Gaussian function, and determines a color filter from the filter size of the color filter and the specified σ value of the Gaussian function (step S610 ).

The luminance component noise removal unit 105 removes noise from the luminance information using the luminance filter determined by the filter determination unit 104 (step S611 ). FIG. 9 is a diagram for explaining the result of noise removal using a luminance filter. The color component noise removal unit 106 removes noise from the color information using the color filter determined by the filter determination unit 104 (step S612).

The image information synthesis unit 107 combines the edge information, noise-removed luminance information, and color information to generate image information in YUV format (step S613 ). FIG. 10 is a diagram for explaining a result obtained by combining edge information and noise-removed luminance information. In the synthesized image information, since edges are clearly expressed as shown in FIG. 10 , an image without blurring of edges and with noise removed is obtained. The image information compressing unit 108 compresses the image information in the YUV format generated by the image information combining unit 107 into a JPEG format (step S614). The image information output unit 109 outputs the image information compressed by the image information compression unit 108 to a memory card or the like (step S615).

In this way, image information is separated into luminance information and color information; edge information is extracted from luminance information; noise removal processing is performed on luminance information and color information, and image information is further synthesized from edge information and noise-removed luminance information and color information . Thereby, edge components can be extracted in advance before noise removal, and edge components can then be combined after noise removal. Thus, an image with image blurring can be compensated for by smoothing edges at the time of noise removal. That is, noise can be effectively removed to maintain high image quality while suppressing edge blurring.

According to the first embodiment, the filter size, σ value, and k value are specified based on imaging conditions, and then the values of the filters are calculated to determine the edge extraction filter. In another example, edge extraction filters may be determined directly from imaging conditions. In this case, a database storing imaging conditions and edge extraction filters in association with each other is provided, and an edge extraction filter corresponding to the imaging conditions is specified. Next, edge information is extracted from the luminance information by using a specified edge extraction filter.

Next, the hardware configuration of a digital camera, that is, an example of an imaging device that performs image processing, is described. Fig. 11 is a block diagram of the hardware configuration of the digital camera according to the first embodiment. As shown in FIG. 11 , light of an object first enters a charge-coupled device (CCD) 3 via an imaging optical system 1 of a digital camera 1000 . Also, the mechanical shutter 2 is located between the imaging system 1 and the CCD 3. Using the mechanical shutter 2, the incident light to the CCD 3 can be cut off. Here, the imaging optical system 1 and the mechanical shutter 2 are driven by a motor driver 6 .

The CCD 3 converts the optical image formed on the imaging surface into an electrical signal to output as analog image data. The image information output from the CCD 3 is denoised by a coherent double sampling (CDS) circuit 4, converted into a digital value by an analog/digital (A/D) converter 5, and then output to an image processing circuit 8.

The image processing circuit 8 uses a synchronous dynamic random access memory (SDRAM) 12 that temporarily stores image data to perform various image processing including YUV conversion, white balance control, contrast correction, edge enhancement, and color conversion. Here, white balance control is image processing to adjust the color density of image information, and contrast correction is image processing to adjust the contrast of image information. Edge enhancement is image processing that adjusts the sharpness of image information, and color conversion is image processing that adjusts the darkness of colors of image information. Also, the image processing circuit 8 subjects image information to signal processing and image processing to be displayed on a liquid crystal display (LCD) 16 .

Also, image information subjected to signal processing and image processing is recorded on the memory card 14 via the compression/decompression circuit 13 . The compression/decompression circuit 13 compresses the image information output from the image processing circuit 8 upon obtaining an instruction from the operation unit 15 , and then outputs the result to the memory card 14 . Also, the compression/decompression circuit 13 decompresses the image information read from the memory card 14 to be output to the image processing circuit 8 .

Also, the CCD 3, the CDS circuit 4, and the A/D converter 5 are controlled by a central processing unit (CPU) 9 via a timing signal generator 7 that generates a timing signal. In addition, the image processing circuit 8, the compression/decompression circuit 13, and the memory card 14 are also controlled by the CPU 9.

In the digital camera 1000, the CPU 9 executes various arithmetic operations according to programs, and integrates therein, for example, a read only memory (ROM) 11 which is a read only memory in which programs and the like are stored, and a random access memory (RAM) 10, The RAM 10 is a freely readable and writable memory having a work area used in various processes and various data storage areas. These elements are connected to each other via a bus.

When the digital camera 1000 performs noise removal processing, the system controller loads a high-sensitivity (sensitive) noise removal program from the ROM 11 to the RAM 10 for execution. The noise removal program acquires the setting of imaging sensitivity and the parameter representing the exposure time at the time of shooting via the system controller. Optimum noise removal setting conditions corresponding to these parameters are read from the ROM 11 for noise removal. The image to be processed is temporarily stored in the SDRAM 12, and the stored image is subjected to noise removal processing.

Next, a noise removal method at the time of shooting will be described. First, the characteristics of high-sensitivity noise will be described. In the digital camera (imaging device) 1000, the amplifier of the circuit is changed to adjust the imaging sensitivity without changing the sensitivity of the CCD 3. When the amount of exposure is small, underexposure occurs. In this case, by increasing the amplification factor of the amplifier, the sensitivity can be increased. However, the noise signal is also amplified at the same time. If the exposure is sufficient, the noise signal is relatively small and not so noticeable. In the case of underexposure, noise is also amplified when the sensitivity is increased by increasing the amplification factor of the amplifier, and high-sensitivity noise becomes conspicuous. This noise is random noise, and color noise occurs even when shooting black and white objects. In order to remove noise of the image information that occurs in this way, the above-described noise removal processing is performed.

The edge extraction filter size database 120, the parameter database 130, the noise removal filter size database 140, and the Gaussian σ value database 150 can be configured with any commonly used storage medium, such as the ROM 11 of the digital camera 1000, a hard disk drive (HDD) , CD-ROM, and memory card.

Also, the image processing program to be executed on the digital camera according to the present embodiment can be stored on a computer connected to a network such as the Internet, and can be provided via network download. Also, the image processing program to be executed on the digital camera according to the present embodiment can be provided or distributed via a network such as the Internet.

Furthermore, the image processing program according to the present embodiment may be provided integrated in a ROM or the like in advance.

The image processing program to be executed on the digital camera according to the present embodiment is recorded on a computer-readable storage medium in an installable or executable format, such as a compact disk read-only memory (CD-ROM), a floppy disk (FD), a recordable CD-R, or Digital Versatile Disc (DVD).

The image processing program to be executed on the digital camera according to the present embodiment has a function including each element (image information acquisition unit, component separation unit, imaging condition acquisition unit, filter determination unit, luminance component edge extraction unit, luminance component noise removal unit, color component noise removal unit, image information synthesis unit, image information compression unit, image information output unit, etc.) module structure. As actual hardware, each unit, namely, image information acquisition unit, component separation unit, imaging condition acquisition unit, filter determination unit, brightness component edge The extraction unit, the brightness component noise removal unit, the color component noise removal unit, the image information synthesis unit, the image information compression unit, the image information output unit, etc. are loaded and generated on the main storage device.

Next, a second embodiment of the present invention will be described. A digital camera as an image processing apparatus according to the second embodiment scales noise-removed luminance information and color information, and combines the scaled luminance information and color information, and edge information to generate image information. Here, differences from the first embodiment will be described.

A structural example of an image processing unit included in a digital camera to which the present invention is applied is explained. FIG. 12 is a block diagram of the structure of the image processing unit 200 according to the second embodiment of the present invention. Image processing unit 200 includes image information acquisition unit 101, component separation unit 102, imaging condition acquisition unit 103, filter determination unit 104, scaling unit 211, brightness component edge extraction unit 110, brightness component noise removal unit 105, color component noise removal Unit 106, inverse scaling unit 212, image information synthesis unit 107, image information compression unit 108, image information output unit 109, edge extraction filter size database 120, parameter database 130, noise removal filter size database 140, Gaussian σ value database 150, and scale factor database 260.

Image information acquisition unit 101, component separation unit 102, imaging condition acquisition unit 103, filter determination unit 104, brightness component edge extraction unit 110, brightness component noise removal unit 105, color component noise removal unit 106, image information synthesis unit 107, The image information compression unit 108, the image information output unit 109, the edge extraction filter size database 120, the parameter database 130, the noise removal filter size database 140, and the structure and function of the Gaussian σ value database 150 are similar to those in the first embodiment Those units will not be described here.

The scaling factor database 260 stores scaling factors for scaling corresponding to imaging conditions for each of brightness information and color information. FIG. 13 is a table for explaining an example of the data structure of the scaling factor database. The scaling factor database 260 stores imaging conditions and scaling factors for brightness information and color information in association with each other.

The scaling factor is a ratio at which the size of luminance information or color information is changed. To reduce the size, set the scaling factor to less than 100%. When scaling is performed with a scaling factor smaller than 100%, the scaled image is smaller than the original image. When filtering a downscaled image, since the downscaled image is smaller than the original image, processing time can be reduced, resulting in faster processing. Also, by reducing the original image, a low-pass filter effect can be obtained.

For example, when the scaling factor is three times in width and three times in length and the size of the noise filter is 5×5, the effect of a noise filter having a size of 15×15 can be obtained. When image information is subjected to 15×15 noise removal filter processing, a large amount of time is required. However, when similar processing is realized by 3×3 scaling processing and noise removal processing using a 5×5 noise removal filter, the processing time can be reduced.

The scaling unit 211 scales the luminance information and color information obtained through the separation by the component separation unit 102 using the scaling factor corresponding to the imaging condition obtained from the scaling factor database 260 .

The inverse scaling unit 212 scales the luminance information from which noise has been removed by the luminance component noise removal unit 105 and the color information from which noise has been removed by the color component noise removal unit 106 using the inverse scaling factor of the scaling unit 211 .

Next, image processing performed by the image processing unit 200 configured as described above will be described. 14A and 14B are image information acquisition unit, component separation unit, imaging condition acquisition unit, filter determination unit, scaling unit, brightness component edge extraction unit, brightness component noise removal unit, color component noise removal unit, reverse scaling unit, A flow chart of the image processing process performed by the image information synthesis unit, the image information compression unit, and the image information output unit.

The image processing procedure according to the second embodiment has parts in common with the flowcharts depicted in FIGS. 6A and 6B, so only different parts will be explained here. For steps S1401 to S1410, refer to the description in FIGS. 6A and 6B , and these steps will not be described here again.

In step S1411, the scaling unit 211 specifies a scaling factor corresponding to the imaging condition from the scaling factor database 160 (step S1411). A scaling factor is specified for each of brightness information and color information. The scaling unit 211 then scales the luminance information and color information separated and obtained by the component separating unit 102 using the respective scaling factors (step S1412).

The luminance component noise removing unit 105 removes noise from the scaled luminance information subjected to the scaling unit 211 using a luminance filter (step S1413 ). The color component noise removal unit 106 removes noise from the scaled color information by the scaling unit 211 using a color filter (step S1414 ).

The inverse scaling unit 212 scales the luminance information from which noise has been removed by the luminance component noise removing unit 105 and the color information from which noise has been removed by the color component noise removing unit 106 , respectively, by a reciprocal scaling factor (step S1415 ).

The image information combining unit 107 combines the edge information, the scaled brightness information and the color information to generate image information in YUV format (step S1416). The image information compressing unit 108 compresses the image information in the YUV format generated by the image information combining unit 107 into the JPEG format (step S1417). The image information output unit 109 outputs the image information compressed by the image information compression unit 108 to a memory card or the like (step S1418).

In this way, even if the size of the noise increases due to an increase in the sensitivity of the camera so that the size of the noise filter must be increased, the luminance information and the color information are scaled for the noise removal process, followed by an inverse scaling factor after the processing zoom. Thereby, the time required for noise removal processing can be reduced, thereby realizing efficient noise removal processing at high speed.

A third embodiment of the present invention will be described below. In the image processing apparatus according to the third embodiment, edge extraction and noise removal of image information are performed in the image processing apparatus instead of the imaging device. Here, only the parts different from the first embodiment will be described.

A structural example of an image processing unit to which the present invention is applied is explained. FIG. 15 is a block diagram of the configuration of an image processing apparatus 300 according to a third embodiment of the present invention. The image processing device 300 includes an image information acquisition unit 301, an image information conversion unit 313, a component separation unit 102, an imaging condition acquisition unit 303, a filter determination unit 104, a luminance component edge extraction unit 110, a luminance component noise removal unit 105, a color component Noise removal unit 106, image information synthesis unit 107, image information conversion unit 314, image information output unit 309, edge extraction filter size database 120, parameter database 130, noise removal filter size database 140, and Gaussian σ value database 150.

Component separation unit 102, filter determination unit 104, brightness component edge extraction unit 110, brightness component noise removal unit 105, color component noise removal unit 106, image information synthesis unit 107, edge extraction filter size database 120, parameter database 130, The structures and functions of the noise removal filter size database 140, and the Gaussian σ value database 150 are similar to those in the first embodiment, and will not be described here.

The image information acquisition unit acquires image information stored in a storage medium or image information transmitted via a network. The image information conversion unit 313 converts the image information acquired by the image information acquisition unit 301 into image information in YUV format.

The imaging condition acquisition unit 303 acquires imaging conditions from the image information acquired by the image information acquisition unit 301 . The image information conversion unit 314 converts the image information in the YUV format generated by the image information synthesis unit 107 into image information in other formats. The image information output unit 309 outputs the image information converted by the image information conversion unit 314 to an HDD or a printer.

Image processing performed by the image processing device configured as described above will be described. 16A and 16B are image information acquisition unit, image information conversion unit, component separation unit, imaging condition acquisition unit, filter determination unit, luminance component edge extraction unit, luminance component noise removal unit, color component noise removal unit, image information synthesis unit, the image information conversion unit, and the flow chart of the image processing process performed by the image information output unit.

The image processing procedure according to the present embodiment is roughly similar to the flowcharts depicted in FIGS. 6A and 6B , so only the different parts will be described here. For steps S1605 to S1614, refer to the description in FIGS. 6A and 6B , and these steps will not be described here again.

The image information acquisition unit 301 acquires image information stored in a storage medium or image information transmitted via a network (step S1601). The image information conversion unit 313 converts the image information acquired by the image information acquisition unit 301 into image information in YUV format (step S1602). For example, when acquired image information is in RGB format, such image information is converted into image information in YUV format by the following conversion equation.

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; Y</span>}\\ \mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; V</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 0.299</span>}& \mathrm{<span\; class="notranslate">\; 0.587</span>}& \mathrm{<span\; class="notranslate">\; 0.114</span>}\\ \mathrm{<span\; class="notranslate">\; 0.5</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.419</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.081</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.169</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.332</span>}& \mathrm{<span\; class="notranslate">\; 0.5</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; G</span>}\\ \mathrm{<span\; class="notranslate">\; B</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

The component separation unit 102 separates the converted image information in YUV format into brightness information and color information (step S1603). The imaging condition acquisition unit 303 acquires imaging conditions from the image information acquired by the image information acquisition unit 301 (step S1604). For example, when the image information is in the Exchangeable Image File Format (Exif), the image information additionally records data of the imaging device, such as manufacturer, model number, imaging sensitivity, and exposure time when the imaging device shoots.

For the description of steps S1605 to S1614, refer to the description of FIGS. 6A and 6B. The image information conversion unit 314 converts the image information in the YUV format generated by the image information synthesis unit 107 into, for example, image information in the RGB format (step S1615). When converting image information in YUV format to image information in RGB format, the following conversion equation is used for conversion.

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; G</span>}\\ \text{<span class="notranslate"> B</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1.402</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; 0</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.174</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.344</span>}\\ \mathrm{<span\; class="notranslate">\; 1.0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1.772</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; Y</span>}\\ \mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; V</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

The image information output unit 309 outputs the image information converted by the image information converting unit 314 to a storage medium or a printer (step S1616).

In this way, even in the image processing apparatus, image information in YUV format is separated into luminance information and color information; edge information is extracted from luminance information; noise is removed from luminance information and color information; and edge information, luminance information and color information. Thereby, noise can be effectively removed to maintain high image quality while suppressing edge blurring. At the same time, noise is removed by using a filter suitable for separating each of the obtained luminance information and color information. Thereby, effective noise removal can be performed in consideration of the characteristics of the human eye.

The scaling processing described in the second embodiment can be incorporated into this embodiment. Thus, even in the image processing apparatus, even if the size of the noise increases due to an increase in imaging sensitivity so that the size of the noise filter must be increased, the luminance information and the color information are scaled for the noise removal processing, and then after the processing in the Reciprocal scaling factor for scaling. Thereby, the time required for noise removal processing can be reduced, thereby realizing efficient noise removal processing at high speed.

Fig. 17 is a block diagram of a hardware configuration of an image processing apparatus according to a third embodiment. The image processing apparatus 300 includes a CPU 24 that centrally controls each element, and the CPU 24 is connected via a bus to a ROM 22 in which a basic input output system (BIOS) and the like are stored, and a ROM 22 in which various data are writably stored and serves as a work area of the CPU. RAM 21, thereby forming a microcomputer. Furthermore, the bus connects an HDD 25 storing a control program, a CD-ROM drive 26 that reads a CD-ROM 28, and an interface (I/F) 23 as an interface for communication with a printer unit or the like.

Predetermined control programs are stored in the CD-ROM 28 depicted in FIG. 17 . The CPU 24 reads the control program stored in the CD-ROM 28 at the CD-ROM drive 26, and then installs the program to the HDD 25. Thereby, various processes described above can be performed. At the same time, image information and the like are stored in the memory card 29 and read by the memory card drive 27 .

As the storage medium, various media such as various optical disks including DVD, various magneto-optical disks, various magnetic disks including floppy disks, and semiconductor memories can be used in addition to CD-ROM or memory card. Meanwhile, the program can be downloaded on a network such as the Internet, and can be installed into the HDD 25. In this case, the storage device in which the program is stored on the server on the transmission side is also a storage medium according to the present invention. Here, the program can work on a predetermined operating system (OS). In this case, a part of various processes to be described further below may be executed by the OS or may be included in predetermined application software such as word processor software, or as part of a set of program files forming an operating system or the like.

Like the first embodiment, the edge extraction filter size database 120, the parameter database 130, the noise removal filter size database 140, and the Gaussian σ value database 150 can be configured with any commonly used storage medium, such as HDD, optical disk, and storage card.

Also, the image processing program to be executed on the image processing apparatus according to the present embodiment may be stored on a computer connected to a network such as the Internet, and may be provided via network download. Also, the image processing program to be executed on the image processing apparatus according to the present embodiment can be provided or distributed via a network such as the Internet.

Furthermore, the image processing program according to the present embodiment may be provided integrated in a ROM or the like in advance.

The image processing program to be executed on the image processing apparatus according to the present embodiment is provided in an installable or executable format recorded on a computer-readable storage medium such as CD-ROM, FD, CD-R, or DVD.

The image processing program to be executed on the image processing apparatus according to the third embodiment has components including each element (image information acquisition unit, image information conversion unit, component separation unit, imaging condition acquisition unit, filter determination unit, luminance component edge Extraction unit, brightness component noise removal unit, color component noise removal unit, image information synthesis unit, image information output unit, etc.) module structure. As actual hardware, as the CPU (processor) reads the image processing program from the storage medium for execution, each unit (i.e., image information acquisition unit, image information conversion unit, component separation unit, imaging condition acquisition unit, filter device determination unit, luminance component edge extraction unit, luminance component noise removal unit, color component noise removal unit, image information synthesis unit, image information output unit, etc.) are loaded and generated on the main storage device.

Next, a fourth embodiment of the present invention will be described. First, a configuration example of an image processing unit included in a digital camera to which the present invention is applied is explained. FIG. 18 is a block diagram of an image processing unit 400 according to a fourth embodiment of the present invention.

The image processing unit 400 includes an image information acquisition unit 101, a high-sensitivity (sensitive) low-resolution image generation unit 415, a component separation unit 402, a color component noise removal unit 406, a brightness component noise removal unit 405, and a brightness component edge extraction unit 410 , a scaling unit 411 , a luminance component synthesis unit 416 , an image information synthesis unit 407 , and an image information output unit 109 . The structures and functions of the image information acquisition unit 101 and the image information output unit 109 are similar to those in the first embodiment, and will not be described here again.

The high-sensitivity low-resolution image generation unit 415 adds the pixel values of a plurality of adjacent pixels in the image information acquired by the image information acquisition unit 101 to calculate one pixel value, thereby generating a high-sensitivity low-resolution image from the acquired image information. image information. For example, when four adjacent pixel values are added, the number of pixels becomes one quarter. However, the amount of light per pixel becomes quadruple, so the sensitivity becomes quadruple. Here, the number of pixels whose pixel values are added is not limited to four. Meanwhile, the shape formed by pixels whose pixel values are added is not limited to a square, but may be any of various polygons, straight lines, or polygonal lines.

The component separation unit 402 separates the high-sensitivity low-resolution image information generated by the high-sensitivity low-resolution image generation unit 415 into brightness information and color information. At the same time, the component separation unit 402 separates the image information acquired by the image information acquisition unit 101 into brightness information and color information. Here, the image information acquired by the image information acquiring unit 101 is low-sensitivity high-resolution image information, with respect to high-sensitivity low-resolution image information generated by adding pixel values.

The color component noise removal unit 406 uses filters stored in the device to remove noise from color information. Here, as the filter, in addition to the filter stored in the device as explained in the first embodiment, a Gaussian smoothing filter can be calculated from the σ value corresponding to the imaging condition, and the calculated filter can be used to obtain from Remove noise from color information.

The luminance component noise removal unit 405 removes noise from luminance information using filters stored in the device. Here, as the filter, in addition to the filter stored in the device as explained in the first embodiment, a Gaussian smoothing filter can be calculated from the σ value corresponding to the imaging condition, and the calculated filter can be used to obtain from Noise removal from brightness information.

The luminance component edge extraction unit 410 extracts edge information from luminance information using filters stored in the device. Here, as the filter, in addition to the filter stored in the device as explained in the first embodiment, a Laplacian of Gaussian (LoG) filter can be calculated from the σ value and the k value corresponding to the imaging condition, And the calculated filter can be used to extract edge information.

The scaling unit 411 scales color information from which noise has been removed by the color component noise removing unit 406 , and performs scaling on luminance information obtained by separation by the component separation unit 402 . For example, the high-sensitivity low-resolution image generation unit 415 adds four pixel values to calculate one pixel value to generate high-sensitivity low-resolution image information, and when the image size is quartered, each The pixel values are quadrupled so that the image has the original image dimensions.

The luminance component synthesis unit 416 combines the luminance information from the scaled luminance information by the scaling unit 411 , the luminance information from which noise has been removed by the luminance component noise removal unit 405 , and the edge information extracted from the luminance component edge extraction unit 410 .

The image information synthesis unit 407 synthesizes image information from the scaled color information by the scaling unit 411 and the luminance information synthesized by the luminance component synthesis unit 416 .

Next, image processing performed by the image processing unit 400 configured as described above will be described. Figure 19 is an image information acquisition unit, a high-sensitivity low-resolution image generation unit, a component separation unit, a color component noise removal unit, a luminance component noise removal unit, a luminance component edge extraction unit, a scaling unit, a luminance component synthesis unit, and an image information A flowchart of the image processing process performed by the synthesis unit and the image information output unit.

First, the image information acquisition unit 101 acquires image information from a temporary storage memory (step S1901). The high-sensitivity low-resolution image generation unit 415 adds adjacent pixel values in the image information to generate high-sensitivity low-resolution image information (step S1902). Thus, the number of pixel values is reduced, but the exposure per pixel is increased. The component separation unit 402 separates the high-sensitivity low-resolution image information into color information (CrCb signal) and luminance information (Y signal) (step S1903). The color component noise removing unit 406 removes noise from the color information obtained by the separation (step S1904). A low-pass filter (for example, a smoothing filter) is used for the noise removal process here. FIG. 20 is a diagram for explaining an example of a smoothing filter having a filter size of 3×3.

The scaling unit 411 scales the noise-removed color information to the original image size (step S1905). The scaling unit 411 scales the luminance information obtained by the separation to the original image size (step S1906). Thus, by separating luminance information obtained from high-sensitivity low-resolution image information generated by adding pixel values and color information obtained from high-sensitivity low-resolution image information by separating and further removing noise has Image dimensions before pixel values are added.

The component separation unit 402 separates the image information acquired by the image information acquisition unit 101 into color information (CrCb signal) and luminance information (Y signal) (step S1907). The luminance component noise removing unit 405 removes noise from the luminance information obtained by separation (step S1908). For the noise removal processing here, like the processing in the color component noise removal unit 406 , a low-pass filter (for example, a smoothing filter shown in FIG. 20 ) is used.

Meanwhile, the luminance component edge extraction unit 410 extracts edge information from the luminance information obtained by the separation (step S1909). Here, for edge extraction processing, a LoG filter is used as an edge extraction filter. The filter can be calculated by using equation (1) above. FIG. 21 is a diagram for explaining an example of an edge extraction filter having a filter size of 5×5. Also, for the edge component extraction processing, the above-mentioned equation (3) is used. FIG. 22 is a diagram for explaining separated luminance information. FIG. 23 is a diagram for explaining results obtained by extracting edges from luminance information. In this way, when an edge is extracted from luminance information as shown in FIG. 22 by using the edge extraction filter as shown in FIG. 21 , an edge component as shown in FIG. 23 is extracted.

The luminance component synthesizing unit 416 synthesizes luminance information from luminance information obtained by separation from high-sensitivity low-resolution image information and scaled, noise-removed luminance information, and edge information (step S1910 ). Fig. 24 is a diagram illustrating a result obtained by combining luminance information and edge information. The edge component is extracted as in FIG. 24 , and combined with brightness information obtained by separation from high-sensitivity low-resolution image information and scaled, and noise-removed brightness information. Thereby, high-quality images can be obtained without reducing the resolution of the images. The image information synthesis unit 407 synthesizes image information from color information obtained by separating and scaling from high-sensitivity low-resolution image information and synthesized luminance information (step S1911 ). The image information output unit 109 outputs the synthesized image information (step S1912). Here, the processing of steps S1907 to S1909 and the processing of steps S1902 to S1906 may be performed simultaneously.

In this way, image information is synthesized from luminance information from high-sensitivity low-resolution image information, noise-removed color information, noise-removed luminance information from high-sensitivity low-resolution image information, and edge information . Thereby, noise can be removed even if the exposure time in imaging is short, thereby obtaining image information having good color reproducibility and white balance, and having an excellent balance between color and brightness of an image.

Brightness information separated from high-sensitivity low-resolution image information is scaled and combined into image information, thereby suppressing noise of the synthesized image. That is, if the luminance value of the image information is used unchanged, since the luminance value of the image information is low, it is necessary to enlarge the luminance value of the image information to increase it. Thereby, noise is also enhanced. On the other hand, combining the luminance information of the high-sensitivity low-resolution image can reduce the scaling factor for the enlargement of the image information. Therefore, the amplification factor of noise is reduced, thereby reducing the noise component of the synthesized image. As a result, the noise of the synthesized image is reduced.

According to the fourth embodiment, an image is captured by an imaging unit, and by using a piece of image information stored in a temporary storage memory, high-sensitivity low-resolution image information and low-sensitivity high-resolution image information are generated. Alternatively, high-sensitivity low-resolution image information and low-sensitivity high-resolution image information may be generated by using two pieces of image information obtained by capturing an image twice by the imaging unit. At this time, two pieces of image information can be generated with the same exposure time. Alternatively, two pieces of image information can be generated with different exposure times, and the image information with long exposure time can be used to generate high-sensitivity low-resolution image information, and the image information with short exposure time can be used to generate low-sensitivity image information. High resolution image information.

Next, the hardware configuration of an example of a digital camera, that is, an imaging device that performs image processing, is described. The hardware structure of the digital camera is similar to that in FIG. 11 . Therefore, referring to FIG. 11 and its description, only the different parts will be described here.

The CCD 3 converts the optical image formed on the imaging surface into an electrical signal to output as analog image information. In the image processing unit 400 described above, image information captured once is used to generate high-sensitivity low-resolution image information and low-sensitivity high-resolution image information. Alternatively, image information may be sequentially output by two exposures, and the output image information may be used. In this case, processing of adding pixel values is performed on image information of one exposure. The image information output from the CCD 3 is removed from the noise component by the CDS circuit 4, converted into a digital value by the A/D converter 5, and then output to the image processing circuit 8. Here, noise removal is performed by a circuit, different from noise removal by image processing.

The image processing circuit 8 uses the SDRAM 12 for temporarily storing image information to perform various image processing including YCrCb conversion, white balance control, contrast correction, edge enhancement, and color conversion. Here, white balance control is image processing to adjust the color density of image information, and contrast correction is image processing to adjust the contrast of image information. Edge enhancement is image processing that adjusts the sharpness of image information, and color conversion is image processing that adjusts the darkness of colors of image information. Also, the image processing circuit 8 subjects image information to signal processing and image processing to be displayed on the liquid crystal LCD 16.

When the digital camera performs high-sensitivity image synthesis processing, the system controller loads the image processing program from the ROM 11 to the RAM 10 for execution. The image processing program accesses the YCrCb image temporarily stored in SDRAM via the system controller to obtain parameters and filters for color component noise removal, luminance component noise removal, and luminance component edge extraction to perform these processes.

The image processing program to be executed on the digital camera according to the present embodiment has components including each element (image information acquisition unit, high-sensitivity low-resolution image generation unit, component separation unit, color component noise removal unit, scaling unit, brightness component edge extraction unit, luminance component noise removal unit, luminance component synthesis unit, image information synthesis unit, image information output unit, etc.) module structure. As actual hardware, as the CPU (processor) reads out the image processing program from the storage medium for execution, each unit (i.e., image information acquisition unit, high-sensitivity low-resolution image generation unit, component separation unit, color component noise removal unit, scaling unit, luminance component edge extraction unit, luminance component noise removal unit, luminance component synthesis unit, image information synthesis unit, image information output unit, etc.) are loaded and generated on the main storage device.

Referring to the drawings, a fifth embodiment will be described. First, a configuration example of an image processing apparatus to which the present invention is applied will be described. FIG. 25 is a block diagram of the structure of an image processing apparatus 500 according to the fifth embodiment.

The image processing apparatus 500 according to this embodiment includes an image information acquisition unit 501, a component conversion unit 517, a component separation unit 402, a color component noise removal unit 406, a scaling unit 411, a luminance component noise removal unit 405, and a luminance component edge extraction unit 410 , a luminance component synthesis unit 416 , an image information synthesis unit 407 , a component conversion unit 518 , an imaging condition acquisition unit 303 , and an image information output unit 309 . Here, the structure and function of the component separation unit 402, the color component noise removal unit 406, the scaling unit 411, the brightness component noise removal unit 405, the brightness component edge extraction unit 410, the brightness component synthesis unit 416, and the image information synthesis unit 407 are similar to those of the first Those units in the four embodiments. In addition, the imaging condition acquisition unit 303 and the image information output unit 309 are similar in structure and function to those in the third embodiment. Therefore, these elements will not be described here.

The image information acquisition unit 501 acquires high-sensitivity low-resolution image information and low-sensitivity high-resolution image information stored in the memory. High-sensitivity low-resolution image information is generated in a manner similar to the fourth embodiment by adding pixel values of a plurality of pixels of image information captured by an imaging unit. In this embodiment, imaging conditions such as the number of pixels whose pixel values at the time of shooting are added are added to high-sensitivity low-resolution image information generated by performing pixel value addition processing at the time of shooting. For example, high-sensitivity low-resolution image information is stored in a memory card of a digital camera. Also, captured image information (low-sensitivity high-resolution image information) to which imaging conditions are added is stored. For example, when the image information is in the Exif format, the imaging conditions added to the image information include the manufacturer, model number, imaging sensitivity, and the number of pixels whose pixel values are added when the imaging device shoots. Here, since adding the pixel values reduces the image size of the high-sensitivity low-resolution image information, the image size of the low-sensitivity high-resolution image information is a normal size.

The component conversion unit 517 converts the image information in the RGB format into the image information in the YCrCb format. The component conversion unit 518 converts the image information of the YCrCb format into the image information of the RGB format.

Next, image processing by the image processing apparatus 500 configured as described above will be described. 26 is an image information acquisition unit, component conversion unit, component separation unit, color component noise removal unit, scaling unit, brightness component noise removal unit, brightness component edge extraction unit, brightness component synthesis unit, image information synthesis unit, and image information Flowchart of the image processing process performed by the output unit.

The process according to the fifth embodiment is roughly similar to the flowchart depicted in FIG. 19 , so only different parts will be explained. For steps S2609 to S2612, refer to the description in FIG. 19 , and these steps will not be described here again.

First, the image information obtaining unit 501 obtains high-sensitivity low-resolution image information in RGB format from the memory (step S2601 ). Since the high-sensitivity low-resolution image information includes imaging conditions, for example, the number of pixels whose pixel values are added at the time of shooting is also obtained. The component converting unit 517 converts the image information in RGB format into image information in YCrCb format (step S2602). The component separation unit 402 separates the high-sensitivity low-resolution image information into color information (CrCb signal) and luminance information (Y signal) (step S2603). The color component noise removing unit 406 removes noise from the color information obtained by the separation (step S2604).

The scaling unit 411 scales the noise-removed color information to the original size using the number of pixels whose pixel values are added obtained as the imaging condition (step S2605 ). The scaling unit 411 scales the luminance information obtained by the separation to the original size using the number of pixels whose pixel values are added obtained as the imaging condition (step S2606 ). Thus, the noise-removed color information obtained by separating luminance information and color information obtained from high-sensitivity low-resolution image information generated by adding pixel values has the original image size before adding pixel values.

The image information acquisition unit 501 acquires low-sensitivity high-resolution image information in RGB format from the memory (step S2607). Since the low-sensitivity high-resolution image information includes imaging conditions, imaging conditions are also obtained. The component conversion unit 517 converts the image information in RGB format into image information in YCrCb format (step S2608). For steps S2609 to S2612, refer to the description of FIG. 19 .

The image information synthesizing unit 407 synthesizes an image mind from the image information of the color information obtained by separation and scaled and the synthesized luminance information (step S2613). The component conversion unit 518 converts the image information of the YCrCb format into the image information of the RGB format (step S2614). The image information output unit 309 outputs the converted image information to a storage medium or a printer (step S2615). Here, the processing of steps S2607 to S2611 may be performed simultaneously with the processing of steps S2601 to S2606.

In this way, even in the image processing apparatus, from luminance information from high-sensitivity low-resolution image information, noise-removed color information, from high-sensitivity low-resolution image information and noise-removed luminance information, and edge information to synthesize image information. Thereby, noise can be removed even when the exposure time is short when imaging, thereby obtaining image information having good color reproducibility and white balance, and having an excellent balance between color and brightness of an image.

According to the fifth embodiment, high-sensitivity low-resolution image information is generated from adding pixel values at the time of shooting, and high-sensitivity low-resolution image information and low-sensitivity high-resolution image information stored in the memory are obtained We started to explain the processing. Alternatively, as in the fourth embodiment, a captured piece of image information may be stored in a memory, and a process of adding pixel values may be performed as image processing to generate high-sensitivity low-resolution image information. Still alternatively, two pieces of image information may be captured by the imaging unit, and processing of adding pixel values may be performed on one of these pieces of image information, followed by the above-mentioned image processing.

The hardware configuration of the image processing apparatus according to the present embodiment is described. Since the hardware configuration of the image processing apparatus is similar to that in FIG. 17 described above, reference is made to FIG. 17 and its description.

In the foregoing, although the present invention has been described by using the first to fifth embodiments, various modifications and improvements can be made to the above-described embodiments. Here, the structures and functions described in the first to fifth embodiments described above can be freely combined.

As described above, according to an aspect of the present invention, edge information extracted in advance is combined to generate image information. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to another aspect of the present invention, edge information is extracted through conventional filtering processing. Therefore, an effect can be achieved that edge extraction processing can be easily performed.

Furthermore, according to still another aspect of the present invention, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to still another aspect of the present invention, edge information is extracted by a filter found based on a value of σ and a value of k corresponding to imaging conditions. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to still another aspect of the present invention, edge information is extracted through a filter corresponding to imaging conditions. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to still another aspect of the present invention, edge extraction suitable for the image quality level can be performed. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to yet another aspect of the present invention, edge extraction suitable for the image quality level can be performed. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to yet another aspect of the present invention, edge extraction suitable for the image quality level can be performed. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to still another aspect of the present invention, noise removal processing is performed after the brightness information and the color information are enlarged or reduced. Therefore, there can be achieved an effect that the time for noise removal processing can be reduced.

In addition, according to still another aspect of the present invention, the sensitivity of image information is increased. Therefore, it is possible to achieve the effect that a high-quality image with excellent color reproduction and white balance can be obtained.

Furthermore, according to yet another aspect of the present invention, noise is removed from high-sensitivity low-resolution color information. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to yet another aspect of the present invention, the previously extracted edge information is combined to generate image information. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to still another aspect of the present invention, the sensitivity of imaged image information can be improved. Therefore, it is possible to achieve the effect that a high-quality image with excellent color reproduction and white balance can be obtained.

Furthermore, according to yet another aspect of the present invention, the previously extracted edge information is combined to generate image information. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained.

Furthermore, according to yet another aspect of the present invention, the previously extracted edge information is combined to generate image information. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained. Meanwhile, noise removal processing is performed after enlarging or reducing brightness information and color information. Therefore, there can be achieved an effect that the time for noise removal processing can be reduced.

Furthermore, according to still another aspect of the present invention, the sensitivity of imaged image information can be improved. Therefore, it is possible to achieve the effect that a high-quality image with excellent color reproduction and white balance can be obtained.

Furthermore, according to yet another aspect of the present invention, the previously extracted edge information is combined to generate image information. Therefore, it is possible to achieve the effect that an image with high quality and less edge blurring can be obtained. Meanwhile, noise removal processing is performed after enlarging or reducing brightness information and color information. Therefore, there can be achieved an effect that the time for noise removal processing can be reduced. Still further, the sensitivity of imaged image information can be improved. Therefore, it is possible to achieve the effect that a high-quality image with excellent color reproduction and white balance can be obtained.

The above and other objects, features, advantages and technical and industrial significance of this invention will be better understood by reading the following detailed description of presently preferred embodiments of the invention, when considered in conjunction with the accompanying drawings.

Claims (10)

1. An image processing apparatus, comprising:

an image information acquisition unit (101) for acquiring image information;

an image component separation unit (102) for separating the image information acquired by the image information acquisition unit (101) into luminance information and color information;

an imaging condition acquisition unit (103) for acquiring an imaging condition corresponding to the image information from the temporary storage memory;

a filter determination unit (104) for determining an edge extraction filter and a noise removal filter corresponding to the imaging conditions;

an edge extraction unit (110) for extracting edge information from the luminance information separated by the image component separation unit (102) using the edge extraction filter determined by the filter determination unit (104) and a k value representing the intensity of edge enhancement;

a luminance noise removing unit (105) for removing noise from the luminance information separated by the image component separating unit (102) using the noise removing filter determined by the filter determining unit (104);

a color noise removing unit (106) for removing noise from the color information separated by the image component separating unit (102) using the noise removing filter determined by the filter determining unit (104); and

an image information synthesizing unit (107) for synthesizing image information based on the edge information extracted by the edge extracting unit (110), the luminance information from which noise is removed by the luminance noise removing unit (105), and the color information from which noise is removed by the color noise removing unit (106).

2. The image processing apparatus according to claim 1, further comprising:

a filter size storage unit for storing imaging conditions indicating an imaging state at the time of imaging of the image information and a size of the filter in association with each other; wherein,

a filter determination unit (104) specifies the size of the filter associated with the imaging condition acquired by the imaging condition acquisition unit (103) from a filter size storage unit (120) to determine the filter.

3. The image processing apparatus according to claim 2, further comprising:

a parameter storage unit (130) for storing an imaging condition indicating an imaging state when image information is imaged, in association with a sigma value and a k value representing an intensity of edge enhancement, which are parameters in a Gaussian Laplace function, as parameters for determining a width of a filter,

a filter determination unit (104) specifies a sigma value and a k value associated with the imaging condition acquired by the imaging condition acquisition unit (103), and determines a filter based on a Gaussian Laplace function defined by the specified sigma value and k value.

4. The image processing apparatus according to claim 1, further comprising:

a filter storage unit for storing an imaging condition indicating an imaging state at the time of imaging the image information and a filter extracting an edge in association with each other; wherein,

a filter determination unit (104) determines a filter associated with the imaging condition acquired by the imaging condition acquisition unit (103).

5. The image processing apparatus according to any one of claims 2 to 4, wherein the imaging condition includes at least one of an exposure time, a temperature at the time of imaging, and an imaging sensitivity.

6. The image processing apparatus according to any one of claims 2 to 4, further comprising:

a scaling unit (211) for scaling the luminance information and the color information obtained by the separation by the image component separation unit (102), wherein,

a luminance noise removal unit (105) removes noise from the luminance information scaled by the scaling unit (211); and is

A color noise removal unit (106) removes noise from the color information scaled by the scaling unit (211).

7. The image processing apparatus according to claim 5, further comprising:

a scaling unit (211) for scaling the luminance information and the color information obtained by the separation by the image component separation unit (102), wherein,

a luminance noise removal unit (105) removes noise from the luminance information scaled by the scaling unit (211); and is

A color noise removal unit (106) removes noise from the color information scaled by the scaling unit (211).

8. An imaging device, comprising:

an imaging unit for capturing an image of a target and outputting image information of the target; and

the image processing apparatus according to any one of claims 1 to 7, wherein

An image information acquisition unit (101) acquires image information from an imaging unit.

9. An image processing method, comprising:

acquiring image information;

separating the acquired image information into luminance information and color information;

acquiring an imaging condition corresponding to the image information from the temporary storage memory;

determining an edge extraction filter and a noise removal filter corresponding to the imaging conditions;

extracting edge information from the separated luminance information using the determined edge extraction filter and a k value representing the intensity of the edge enhancement;

luminance noise removal including removing noise from the separated luminance information using the determined noise removal filter;

color noise removal including removing noise from the separated color information using the determined noise removal filter; and

image information is generated based on the extracted edge information, the luminance information from which the noise is removed, and the color information from which the noise is removed.

10. An image processing method, comprising:

acquiring image information;

separating the acquired image information into luminance information and color information;

acquiring an imaging condition corresponding to the image information from the temporary storage memory;

determining an edge extraction filter and a noise removal filter corresponding to the imaging conditions;

extracting edge information from the separated luminance information using the determined edge extraction filter and a k value representing the intensity of the edge enhancement;

scaling the separated luminance information and color information;

luminance noise removal including removing noise from the scaled luminance information using the determined noise removal filter;

color noise removal including removing noise from the scaled color information using the determined noise removal filter; and is

Image information is generated based on the extracted edge information, the luminance information from which the noise is removed, and the color information from which the noise is removed.

Images