KR102628938B1 - Image processing apparatus - Google Patents

Abstract

An image processing device according to an embodiment of the present technology includes a pixel window extraction unit that extracts a pixel window of size M×M (M is an odd number of 3 or more) from image data and outputs Bayer data of the extracted pixel window, the Bayer A low-band calculation unit that calculates low-band values for a center pixel located at the center of the pixel window using data, and a green pixel included in the center pixel and the pixel window using the Bayer data and the low-band values. A chroma component difference calculation unit that calculates the chroma component difference between the chroma components, a gradient calculation unit that calculates a gradient for the center pixel using the Bayer data, and a correction parameter corresponding to the gradient according to a preset function. It may include a damping unit that generates and a fringe reduction unit that generates corrected Bayer data for the center pixel using the low band value for the green pixels, the chroma component difference, and the correction parameter.

Description

Image processing device {IMAGE PROCESSING APPARATUS}

The present invention relates to an image processing device capable of removing color fringe from captured images.

In image processing devices that require high-quality images, technology to generate high-quality images is very important. In these image processing devices, when the quality of the lens used in the camera deteriorates or a wide-angle lens is used, a specific color (e.g., a specific color (e.g., For example, a phenomenon where purple color appears occurs. This phenomenon of purple color is called purple fringe or color fringe.

In the case of cameras that generate HDR (High Dynamic Range) images by combining multiple exposure images, the purple fringe phenomenon that occurs in each exposure image can be synthesized into one HDR image and appear as various forms of purple fringe phenomenon. there is.

Several methods have been proposed to remove this purple fringe, but they are complex to implement and may cause problems such as distortion of colors other than the purple fringe.

The present invention seeks to provide an image processing device capable of removing color fringe from captured images.

An image processing device according to an embodiment of the present invention includes a pixel window extractor that extracts a pixel window of size M×M (M is an odd number of 3 or more) from image data and outputs Bayer data of the extracted pixel window, the Bayer A low-band calculation unit that calculates low-band values for a center pixel located at the center of the pixel window using data, and a green pixel included in the center pixel and the pixel window using the Bayer data and the low-band values. A chroma component difference calculation unit that calculates the chroma component difference between the chroma components, a gradient calculation unit that calculates a gradient for the center pixel using the Bayer data, and a correction parameter corresponding to the gradient according to a preset function. It may include a damping unit that generates and a fringe reduction unit that generates corrected Bayer data for the center pixel using the low band value for the green pixels, the chroma component difference, and the correction parameter.

An image processing device according to another embodiment of the present invention extracts pixel windows of a preset size from first exposure image data and extracts low band values for the center pixel from the extracted pixel windows and chroma between the center pixel and green pixels. A first color fringe correction unit that calculates component differences and corrects color fringe for the first exposure image data, extracts pixel windows of a preset size from the second exposure image data, and rows for the center pixel from the extracted pixel windows. A second color fringe correction unit that corrects color fringe for the second exposure image data by calculating band values and chroma component differences between the center pixel and the green pixel, and a first exposure corrected in the first color fringe correction unit. It may include an image synthesis unit that generates an HDR image by combining image data and second exposure image data corrected by the second color fringe correction unit.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

The present invention can improve image quality by more easily reducing or eliminating color fringe.

1 is a block diagram briefly showing the configuration of an image processing device according to an embodiment of the present invention.
Figure 2 is a diagram illustrating a 5×5 pixel window according to an embodiment of the present invention.
FIG. 3A is a diagram illustrating low-band component estimates for a center pixel and pixels having the same color as the center pixel in a pixel window.
FIG. 3B is a diagram illustrating low-band component estimates for pixels located adjacent to the center pixel in the horizontal direction in a pixel window and pixels having the same color as the pixel.
FIG. 3C is a diagram illustrating low-band component estimates for pixels located adjacent to the center pixel in the vertical direction in a pixel window and pixels having the same color as the pixel.
FIG. 3D is a diagram illustrating low-band component estimates for pixels located adjacent to the center pixel in the diagonal direction in a pixel window and pixels having the same color as the pixel.
Figure 4 is a table showing the relationship between pixel positions and low band values.
FIGS. 5A to 5D are diagrams exemplarily showing the positional relationship of a 2×2 Bayer pattern according to a change in the center pixel.
Figure 6 is a diagram illustrating the gradient (edge size) for four directions (horizontal, vertical and diagonal directions) at the location of the center pixel in a pixel window.
7 is a graph showing the relationship between the gradient value (G _rad ) and the damp factor (dampfactor).
FIG. 8 is a flowchart for explaining the operation of the image processing device of FIG. 1.
FIG. 9 is a diagram illustrating a 5×5 pixel window extracted for color fringe correction for a blue pixel (B) located at a certain position in a pixel array in which RGB pixels are arranged in a Bayer pattern.
FIGS. 10A and 10B are diagrams illustrating a 5×5 pixel window for a blue pixel following the blue pixel in FIG. 9.
Figure 11 is a block diagram briefly showing the configuration of an image processing device according to another embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, when describing embodiments of the present invention, if detailed descriptions of related known configurations or functions are judged to impede understanding of the embodiments of the present invention, the detailed descriptions will be omitted.

1 is a block diagram briefly showing the configuration of an image processing device according to an embodiment of the present invention.

Referring to FIG. 1, the image processing device includes a pixel window extraction unit 10, a low band calculation unit 20, a chroma disparity calculation unit 30, and a gradient calculation unit. ) It may include a crystal unit 40, a damping unit 50, and a fringe reduction unit 60.

The pixel window extractor 10 extracts pixel values (M x M Bayer data) of pixels arranged in an M can do. Hereinafter, pixels arranged in an M×M format are collectively referred to as a pixel window, and M may include an odd number of 3 or more. For example, the pixel window extractor 10 may sequentially extract pixel windows of size M×M for blue color pixels and red color pixels from an image.

FIG. 2 is a diagram illustrating a pixel window according to an embodiment of the present invention. The pixel window extractor 10 extracts the blue pixels and red pixels in the captured image with the corresponding pixels as the center. As shown, the pixel values (Bayer data) of 25 pixels arranged adjacently in a 5×5 format can be extracted and output. That is, the pixel window extractor 10 may extract a pixel window for each pixel according to a preset order for all blue pixels and red pixels. At this time, in the case of pixels located in the boundary area of the image, all pixels of size M×M may not exist around the pixel. In such cases, pixel windows for the corresponding pixels can be created through pixel mirroring, etc.

Hereinafter, for convenience of explanation, the description will be made assuming that 5×5 pixels arranged in a Bayer pattern as shown in FIG. 2 are pixel windows. The pixel window extractor 10 may provide M×M Bayer data (pixel values) of the extracted pixel window to the low band calculator 20.

Since the color fringing phenomenon generally appears in the form of the blue or red component having a larger value than the green component, the color fringe can be reduced by keeping the pixel values of the green pixels the same and lowering the pixel values of the blue and red pixels. You can. Therefore, this embodiment illustratively explains the case of extracting pixel windows only for blue pixels and red pixels.

For each pixel window provided from the pixel window extraction unit 10, the low-band calculation unit 20 uses the pixel values (Bayer data) of pixels included in the pixel window to determine the center pixel of the pixel window. Low band values (GRL, RL, GBL, BL, YL) can be calculated.

For example, FIGS. 3A to 3D are diagrams illustrating low-band component estimates according to the positions of pixels in a 5×5 pixel window.

The low-band calculation unit 20 uses the Bayer data of the provided pixel window to generate four estimated values (XL[0], XL[1], XL[2], XL[3]) as shown in FIGS. 3A to 3D. can be calculated.

To this end, the low-band calculation unit 20 divides the pixel window into four groups as shown in FIGS. 3A to 3D according to the positions of the corresponding pixels, and determines the pixel of the corresponding pixel according to the degree of proximity to the center pixel within each group. By assigning different weights to the values, the estimated values (XL[0], XL[1], XL[2], XL[3]) for each group can be calculated.

For example, the low band calculation unit 20 multiplies the original pixel value by 36 for the center pixel, and multiplies the original pixel value by 36 for pixels located in the vertical and horizontal directions from the center pixel and having the same color as the center pixel. is multiplied by 6, and for pixels located diagonally from the center pixel and having the same color as the center pixel, the original pixel value is multiplied by 1 to calculate the low-band component estimate for XL[0] as shown in Figure 3a. . In addition, the low-band calculation unit 20 multiplies the original pixel value by 6 for pixels (first adjacent pixels) located adjacent to the center pixel in the horizontal direction, and the remaining pixel value having the same color as the first adjacent pixel For pixels, the original pixel value can be multiplied by 1 to calculate the low-band component estimate for XL[1], as shown in Figure 3b.

In addition, the low-band calculation unit 20 multiplies the original pixel value by 6 for pixels located adjacent to the center pixel in the vertical direction (second adjacent pixels), and the remaining pixel value having the same color as the second adjacent pixel For pixels, the original pixel value can be multiplied by 1 to calculate the low-band component estimate for XL[2], as shown in Figure 3c. In addition, the low band calculation unit 20 multiplies the original pixel value by 1 for pixels located adjacent to the center pixel in the diagonal direction (third adjacent pixels) to obtain the row for XL[3] as shown in FIG. 3D. Band component estimates can be calculated.

At this time, based on a 2×2 Bayer pattern including red (R), first green (GR), second green (GB), and blue (B), first green (GR) and second green (GB) can be classified into different colors. For example, when the first adjacent pixels are pixels of the first green (GR) color, pixels having the same color as the first adjacent pixels correspond only to pixels of the first green (GR) color and are pixels of the second green (GB) color. ) Colored pixels may not be included.

When the low-band calculation unit 20 calculates low-band component estimates (XL[0], XL[1], XL[2], XL[3]) for each pixel window as shown in FIGS. 3A to 3D, To match the levels of the estimates (XL[0], XL[1], XL[2], XL[3]) to the same size, the estimates (XL[1], XL[2], XL[3]) Additionally, weights can be multiplied. For example, in the above-described embodiment, the sum of the weights multiplied by the corresponding pixel values to obtain the estimated value (XL[0]) is 64 (=36+6+6+6+6+1+1+1+ 1), the sum of the weights multiplied by the corresponding pixel values to obtain the estimated values (XL[1], XL[2], XL[3]) is each 16 (=6+6+1+1+1+ 1), 16(=6+6+1+1+1+1), 4(=1+1+1+1). Therefore, in order to match the level of the estimates (XL[1], XL[2], XL[3]) with the estimate (XL[0]), the estimates (XL[1], XL[2]) are set to 4. Multiply, and the estimated value (XL[3]) can be multiplied by 16.

Additionally, the low-band calculation unit 20 may check which pixel is the center pixel of the corresponding pixel window and calculate low-band values (GRL, RL, GBL, BL, YL) for the center pixel.

At this time, the low band values (GRL, RL, GBL, BL) are as shown in the table in FIG. 4, assuming that the center pixel of the corresponding pixel window is a pixel located at the top-left of the 2×2 Bayer pattern. , the value can be determined according to the positional relationship of pixels in the 2×2 Bayer pattern.

Figure 4 is a table illustrating low band values for each position of the center pixel. The pixel positions (GBRG, GRBG, RGGB, BGGR) in Figure 4 are the center pixel of the extracted pixel window in the 2×2 Bayer pattern. Assuming that it is located in the top-left, it can represent the positional relationship (array form) of the pixels (R, GR, GB, B) in the 2×2 Bayer pattern.

For example, the pixel position (BGGR) indicates the case where the blue pixel (B) is the center pixel of the pixel window. As shown in Figure 5a, the blue pixel (B) is located at the top-left of the 2×2 Bayer pattern. left), the 2×2 Bayer pattern can be a pattern in which pixels (R, GR, GB, B) are arranged in BGGR order when viewed in the order of ①②③④. Additionally, the pixel position (RGGB) indicates the case where the red pixel (R) is the center pixel of the pixel window. As shown in FIG. 5b, the red pixel (R) is located at the top-left of the 2×2 Bayer pattern. When located, the 2×2 Bayer pattern can be a pattern in which the pixels (R, GR, GB, B) are arranged in RGGB order when viewed in the order of ①②③④.

Likewise, the pixel positions GBRG and GRBG are the case where the green pixel GB and the green pixel GR are the center pixels of the pixel window, respectively, as shown in FIGS. 5C and 5D. This may be the case where (GR) is located at the top-left of the 2×2 Bayer pattern.

In the table of Figure 4, X[0] can be the sum of the estimated values of XL[0] in Figure 3a, and It can be a sum of values. Additionally, X[2] can be the sum of the estimated values of XL[2] in Figure 3c multiplied by 4, and It can be the sum of the multiplied values.

The low-band calculation unit 20 calculates four rows based on the low-band component values (X[0], Band values (GRL, RL, GBL, BL) can be calculated. For example, if the blue pixel (B) is the center pixel of the pixel window, the pixel position corresponds to BGGR as shown in Figure 5A, and the low band values (GRL, RL, BL, GBL) corresponding to BGGR are respectively shown in Figure 5A. Based on the table in 4, it can be matched with X[2], X[3], X[0], and X[1]. The low-band calculation unit 20 divides each of the low-band component values (X[2], X[3], X[0], and , BL, GBL). That is, the values of X[0], 0], X[1], X[2], and

In this way, the low-band calculation unit 20 calculates and determines the low-band component estimate values (XL[0], XL[1], XL[2], XL[3]) and determines what color the center pixel of the corresponding pixel window is. If you know whether it is a pixel, you can calculate low band values (GRL, RL, GBL, BL) for the center pixel.

The low band value (YL) may mean a low band value for the brightness (luminance) of the corresponding center pixel. This low band value (YL) can be obtained using GRL, RL, GBL, and BL.

For example, the low band calculation unit 20 can calculate the low band value (YL) using the formula below.

The chroma component difference calculation unit 30 calculates the pixel values (5×5 Bayer data) of the pixel window output from the pixel window extraction unit 10 and the low band values (GRL, RL) received from the low band calculation unit 20. , GBL, BL, YL), the difference in chroma components between the center pixel of the corresponding pixel window and the green pixels included in the corresponding pixel window can be obtained.

For example, when the center pixel of a pixel window is a blue pixel (B), the chroma component difference calculation unit 30 calculates the low band value (BL) of the center pixel (B) for the pixel window and the green pixels (GB). , GR), find the difference (Cg _gap ) between the average low band values {GL = (GRL+GBL)/2}, and calculate the original pixel value (raw Bayer data, C _bayer ) of the center pixel (B) and the low band value. The difference (C _gap ) of (BL) can be obtained. When the center pixel of the pixel window is a red pixel (R), the chroma component difference calculation unit 30 calculates the low band value (RL) of the center pixel (R) and the green pixels (GB, GR) for the pixel window. Find the difference (Cg _gap ) between the average low band values {GL = (GRL+GBL)/2}, and calculate the difference between the original pixel value (raw Bayer data, C _bayer ) of the center pixel (R) and the low band value (RL). You can find the difference (C _gap ).

The chroma component difference calculation unit 30 can calculate the chroma component differences (Cg _gap , C _gap ) using the formula below.

Here, CL is the low band value of the center pixel, which can be BL when the center pixel is a blue pixel and RL when the center pixel is a red pixel.

The gradient calculation unit 40 may obtain a gradient for the center pixel using pixel values (5×5 Bayer data) of the pixel window output from the pixel window extraction unit 10. For example, Figure 6 is a diagram illustrating the gradient (edge size) for four directions (horizontal, vertical, and diagonal directions) at the location of the center pixel in a 5×5 pixel window. The gradient calculation unit 40 uses the SOBEL algorithm to calculate the size of the edge in four directions at the position of the center pixel as shown in FIG. 6 and converts it into gradient values (Gx, Gy, It can be done with Gn, Gz). At this time, Gx may represent the gradient value in the horizontal direction, Gy may represent the gradient value in the vertical direction, and Gn and Gz may represent gradient values in the diagonal directions. In this embodiment, the gradient is calculated using the Sobel algorithm, but the size of the gradient can also be calculated using other algorithms such as prewitt or LoG (Laplacian of Gaussian).

The gradient calculation unit 40 calculates the magnitude of the absolute values (|Gx|, |Gy|, |Gn|, |Gz|) of the gradient values (Gx, Gy, Gn, Gz) in the four directions as shown in the formula below. By comparing , the largest value can be determined as the gradient value (G _rad ) of the center pixel.

Here, |Gx|, |Gy|, |Gn|, and |Gz| may mean the sum of the absolute values of each value in Gx, Gy, Gn, and Gz of FIG. 6, respectively.

The damping unit 50 may generate a correction parameter for adjusting the attenuation strength of the color fringe based on the gradient value (G _rad ) output from the gradient calculation unit 40 . In this embodiment, a damp factor may be used as a correction parameter.

For example, Figure 7 is a graph showing the relationship between the gradient value (G _rad ) and the damp factor. As shown in Figure 7, when the gradient value is small, the damping unit 50 increases the damp factor to increase the color fringe attenuation strength. If the gradient value is large, the damp factor can be decreased to increase the color fringe attenuation strength.

This damp factor can be adjusted (designed) to suit the characteristics of the image sensor. For example, the position of fading in FIG. 7 and the slope of the damp factor after fading can be adjusted to enable tuning according to the color fringe intensity of the image sensor or the nature of the color fringe. The graph of the damp factor may use a Gaussian weight function.

The fringe reduction unit 60 calculates the chroma component differences (Cg _gap , C _gap ) provided from the chroma component difference calculation unit 30, the low band value (GL) for the green pixels (GB, GR), and the damping unit 50. ) can be used to generate a pixel value (corrected Bayer data) (C _correct ) with the color fringe for the center pixel reduced or removed using the damp factor provided from ). For example, the fringe reduction unit 60 can correct the pixel value of the center pixel using the formula below.

When color fringe reduction for the corresponding center pixel is completed, the fringe reduction unit 60 may transmit a signal notifying this fact to the pixel window extraction unit 10.

FIG. 8 is a flowchart for explaining the operation of the image processing device of FIG. 1.

When the image sensor captures an object and image data for the object is acquired, the acquired data may be transmitted to the pixel window extractor 10. At this time, the Bayer data may be the pixel value (raw Bayer data) of each pixel obtained through photoelectric conversion in the image sensor.

The pixel window extractor 10 can extract a 5×5 pixel window with the pixel as the center pixel for a pixel to be corrected (color fringe reduction or removal) from the provided raw Bayer data (S810 ).

For example, Figure 9 illustrates a 5×5 pixel window extracted for color fringe correction for a blue pixel (B) at a gray position in a pixel array in which RGB pixels are arranged in a Bayer pattern. As shown in the drawing, the pixel window extractor 10 can extract and output pixel values (raw Bayer data) of 25 pixels within the area indicated by a dotted line in FIG. 9.

The extracted Bayer data of the 5×5 pixel window may be provided to the low band calculator 20, the chroma component calculator 30, and the gradient calculator 40.

The low band calculation unit 20 uses the Bayer data (pixel values) of the pixel window provided from the pixel window extraction unit 10 to calculate the row for the center pixel (GB pixel shown in gray in FIG. 9) of the pixel window. Band values (GRL, RL, GBL, BL, YL) can be calculated (S822).

To this end, the low-band calculation unit 20 calculates low-band component estimates (XL[0], XL[1], XL[2], XL[3]) for the corresponding pixel window as shown in FIGS. 3A to 3D. You can. Next, the low band calculation unit 20 can determine what color pixel is the center pixel of the corresponding pixel window. As shown in Figure 9, when the center pixel is a blue pixel (B), when the B pixel is located at the top-left of the 2 × 2 Bayer pattern, the pixel position of the 2 × 2 Bayer pattern is in Figure 5a. It corresponds to BGGR as follows.

Therefore, the low band calculation unit 20 divides the values (X[2], These can be set to corresponding low band values (GRL=X[2], RL=X[3], GBL=X[0], BL=X[1]).

When the low band values (GRL, RL, GBL, BL) are obtained, the low band calculation unit 20 can calculate the low band value (YL) using Equation 1 described above. The obtained low band values (GRL, RL, GBL, BL, YL) may be transmitted to the chroma component difference calculator 30.

The chroma component difference calculation unit 30 calculates the pixel value (C _bayer ) of the center pixel (B) among the pixel values of the pixel window provided from the pixel window extraction unit 10 and the row provided from the pixel window extraction unit 10. The difference in chroma components (Cg _gap , C _gap ) between the blue pixel and the green pixel can be obtained using the band values (GRL, RL, GBL, BL, YL) (S832).

For example, the chroma component difference calculation unit 30 calculates the low band value (BL) of the center pixel (B) and the low band value of the green pixels (GB, GR) {GL = ( GRL+GBL)/2}, the difference (Cg _gap ) can be obtained, and the difference (C _gap ) between the raw pixel value (C _bayer ) of the center pixel (B) and the low band value (BL) can be obtained.

If the center pixel of the pixel window is a red pixel (R), the chroma component difference calculator 30 calculates the low band value (RL) of the center pixel (R) and the low band of the green pixels (GB, GR). You can find the difference (Cg _gap ) between the values {GL = (GRL+GBL)/2}, and the difference (C _gap ) between the raw pixel value (C _bayer ) of the center pixel (R) and the low band value (RL). there is.

The color fringe phenomenon generally appears in the form where the blue or red component has a larger value than the green component. Therefore, color fringe can be reduced by keeping the pixel value of the green pixel the same and lowering the value of the blue pixel or red pixel.

The chroma component difference _{calculation unit 30 transmits the values for the chroma component differences (Cg gap, C gap ) and the} low band values (GL) for the green pixels to the fringe reduction unit 60. You can.

While steps S822 and S832 are in progress, the gradient calculation unit 40 uses Bayer data (pixel values) of the pixel window provided from the pixel window extraction unit 10 to calculate a gradient (G _rad ) for the center pixel. ) is obtained (S824), and the damping unit 50 may generate a damp factor corresponding to the gradient value (G _rad ) output from the gradient calculation unit 40 (S834).

For example, as shown in FIG. 6, the gradient calculation unit 40 calculates the edge size for four directions (horizontal, vertical, and diagonal directions) at the position of the center pixel and calculates the edge size for the corresponding directions. You can obtain gradient values (Gx, Gy, Gn, Gz). At this time, Gx may represent the gradient value in the horizontal direction, Gy may represent the gradient value in the vertical direction, and Gn and Gz may represent gradient values in the diagonal directions.

This gradient can be calculated using the already known Sobel algorithm or using an algorithm such as prewitt or LoG (Laplacian of Gaussian).

The gradient calculation unit 40 compares the sizes of the absolute values (|Gx|, |Gy|, |Gn|, |Gz|) of the gradient values (Gx, Gy, Gn, Gz) in the four directions and selects the largest The value can be determined as the gradient value ( _Grad ) of the center pixel. The gradient calculation unit 40 may transmit the determined gradient value (G _rad ) to the damping unit 50.

The damping unit 50 may obtain a damping factor corresponding to the gradient value ( _Grad ) based on the function graph shown in FIG. 7 and transmit the value to the fringe reduction unit 60.

The fringe reduction unit 60 calculates the chroma component differences (Cg _gap , C _gap ) provided from the chroma component difference calculation unit 30, the low band value (GL) for the green pixels (GB, GR), and the damping unit 50. ) can be used to generate and output the corrected pixel value (corrected Bayer data) for the center pixel (S840).

For example, the fringe reduction unit 60 may generate a corrected pixel value for the center pixel using Equation 4 described above.

When correction for the corresponding center pixel is completed, the fringe reduction unit 60 may transmit a signal notifying this fact to the pixel window extraction unit 10.

When a signal indicating that correction (color fringe reduction) has been completed is received from the fringe reduction unit 60, the pixel window extractor 10 can check whether there is a pixel to be corrected next according to a preset order (S850) ).

If the next pixel to be corrected exists, the pixel window extractor 10 may re-extract a pixel window with that pixel as the center pixel. For example, the pixel window extractor 10 extracts a pixel window with the next horizontally adjacent blue pixel as the center pixel, as shown in FIG. 10A, or the next vertically adjacent blue pixel, as shown in FIG. 10B. A pixel window with as the center pixel can be extracted.

Correction for the corresponding center pixel can also be performed on the extracted new pixel windows by performing the above-described steps S822 to S840.

Additionally, the above-described operations for color fringe correction (reduction or removal) may be performed on both blue pixels and red pixels, or may be selectively performed on only one color among blue pixels and red pixels.

When it is desired to produce an HDR (High Dynamic Range) image by compositing multiple exposure images, the image processing device of the present embodiment first performs the color fringe correction described above for each exposure image to be synthesized, and then performs the corrected exposure images. HDR video can be produced by compositing.

Figure 11 is a block diagram briefly showing the configuration of an image processing device according to another embodiment of the present invention.

Referring to FIG. 11 , the image processing device may include a plurality of color fringe correction units 110 and 120 and an image synthesis unit 200.

The plurality of color fringe correction units 110 and 120 may each include the components shown in FIG. 1 described above. At this time, image data (Bayer data) input to each color fringe correction unit 110 and 120 may be different exposure images for HDR synthesis. That is, each color fringe correction unit 110 and 120 may output an exposure image in which the color fringe has been corrected by performing the processes of FIG. 8 described above for different exposure images.

The image synthesis unit 200 may generate an HDR image by combining exposure images output from the plurality of color fringe correction units 110 and 120. Any of the conventional HDR image synthesis methods may be used to produce an HDR image by synthesizing a plurality of exposure images.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations will be possible to those skilled in the art without departing from the essential characteristics of the present invention.

Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but rather to explain it, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

Claims (14)

a pixel window extraction unit that extracts a pixel window of size M×M (M is an odd number of 3 or more) from the image data and outputs Bayer data of the extracted pixel window;
a low-band calculation unit that calculates low-band component estimates for Bayer data of the pixel window and calculates low-band values for the center pixel based on the low-band component estimates and a position of the center pixel of the pixel window;
a chroma component difference calculator that calculates a chroma component difference between the center pixel and green pixels included in the pixel window using the Bayer data and the low band values;
a gradient calculation unit that calculates a gradient for the center pixel using the Bayer data;
a damping unit that generates a correction parameter corresponding to the gradient according to a preset function; and
An image processing device including a fringe reduction unit that generates corrected Bayer data for the center pixel using the low band value for the green pixels, the chroma component difference, and the correction parameter. The method of claim 1, wherein the pixel window extractor
An image processing device, characterized in that the pixel window is extracted for at least one color among blue color pixels and red color pixels from the image data. delete The method of claim 1, wherein the low band calculation unit
An image characterized in that the pixel window is divided into four groups according to the positions of the pixels, and within each group, different weights are assigned depending on the degree of proximity to the central pixel to calculate low-band component estimates for each group. processing unit. The method of claim 4, wherein the low band calculation unit
Classifying the center pixel of the pixel window and pixels having the same color as the center pixel into a first group,
Classifying pixels having the same color as the first adjacent pixels into a second group based on the first adjacent pixels located adjacent to the center pixel in the horizontal direction and a 2×2 Bayer pattern,
Classifying pixels having the same color as the second adjacent pixels into a third group based on the second adjacent pixels located adjacent to the center pixel in the vertical direction and a 2×2 Bayer pattern,
An image processing device, characterized in that classifying third adjacent pixels located adjacent to the center pixel in a diagonal direction into a fourth group. The method of claim 5, wherein the low band calculation unit
In the first group, a first weight is given to the center pixel, a second weight smaller than the first weight is given to pixels located vertically and horizontally from the center pixel, and pixels located in a diagonal direction from the center pixel A third weight smaller than the second weight is given to the pixels located in,
assigning the second weight to the first adjacent pixels in the second group and assigning the third weight to the remaining pixels,
In the third group, the second weight is given to the second adjacent pixels and the third weight is given to the remaining pixels,
An image processing device characterized in that the third weight is given to the third adjacent pixels in the fourth group. The method of claim 4, wherein the low band calculation unit
The levels of the low-band component estimates are adjusted to the same size and then added for each group to generate four low-band component values,
Assuming that the center pixel of the extracted pixel window is a pixel located at the top-left of the 2×2 Bayer pattern, based on the arrangement form of the pixels included in the 2×2 Bayer pattern, the four low bands The component values are divided by the sum of the weights and the first low band value (BL) for the blue pixel, the second low band value (RL) for the red pixel, and the third low band value (GBL) for the first green color. and matching a fourth low band value (GRL) to the second green color. The method of claim 7, wherein the low band calculation unit
An image processing device, wherein a fifth low band value (YL) for the brightness of the center pixel is obtained using the first to fourth low band values. The method of claim 7, wherein the chroma component difference calculation unit
Obtaining the difference between the low band value of the center pixel and the average low band value (GL) of the first and second green pixels,
An image processing device characterized in that the difference between the original pixel value of the center pixel and the low band value is obtained. The method of claim 1, wherein the gradient calculation unit
Calculate gradient values for each direction in the horizontal, vertical, and diagonal directions at the center pixel,
An image processing device that compares the sizes of the gradient values for each direction and determines the largest value as the gradient of the center pixel. The method of claim 10, wherein the damping unit
An image processing device, characterized in that generating a correction parameter having a size inversely proportional to the gradient size of the center pixel. First pixel windows of a preset size are extracted from the first exposure image data, low band values for the center pixel and chroma component differences between the center pixel and green pixels are obtained from the extracted first pixel windows, and the first pixel windows are extracted. a first color fringe correction unit that corrects color fringe for exposed image data;
Second pixel windows of a preset size are extracted from the second exposure image data, low band values for the center pixel and chroma component differences between the center pixel and green pixels are obtained from the extracted second pixel windows, and the second pixel window is extracted from the second exposure image data. a second color fringe correction unit that corrects color fringe for exposed image data; and
An image synthesis unit that generates an HDR image by combining first exposure image data corrected by the first color fringe correction unit and second exposure image data corrected by the second color fringe correction unit,
The first color fringe correction unit calculates low-band component estimates for Bayer data of the first pixel windows, and calculates low-band values for the center pixel based on the calculated low-band component estimates and the location of the center pixel. ,
The second color fringe correction unit calculates low-band component estimates for Bayer data of the second pixel window, and calculates low-band values for the center pixel based on the calculated low-band component estimates and the location of the center pixel. Image processing device. The method of claim 12, wherein the first color fringe correction unit
a first pixel window extractor that extracts the first pixel window of size M×M (M is an odd number of 3 or more) from the first exposure image data and outputs Bayer data of the extracted first pixel window;
a first low band calculation unit that calculates low band values for a center pixel located at the center of the first pixel window using the Bayer data;
a first chroma component difference calculator that calculates a chroma component difference between a center pixel of the first pixel window and green pixels included in the first pixel window using the Bayer data and the low band values;
a first gradient calculation unit that calculates a gradient for the center pixel using the Bayer data;
a first damping unit that generates a correction parameter corresponding to the gradient according to a preset function; and
An image processing device including a first fringe reduction unit generating correction Bayer data for a center pixel using the low band value for the green pixels, the chroma component difference, and the correction parameter. The method of claim 12, wherein the second color fringe correction unit
a second pixel window extractor that extracts the second pixel window of size M×M (M is an odd number of 3 or more) from the second exposure image data and outputs Bayer data of the extracted second pixel window;
a second low-band calculation unit that calculates low-band values for a center pixel located at the center of the second pixel window using the Bayer data;
a second chroma component difference calculator that calculates a chroma component difference between a center pixel of the second pixel window and green pixels included in the second pixel window using the Bayer data and the low band values;
a second gradient calculator that calculates a gradient for the center pixel using the Bayer data;
a second damping unit that generates a correction parameter corresponding to the gradient according to a preset function; and
An image processing device comprising a second fringe reduction unit generating correction Bayer data for the center pixel using the low band value for the green pixels, the chroma component difference, and the correction parameter.

Images