KR101901602B1 - Apparatus and method for noise removal in a digital photograph - Google Patents

Abstract

The present invention relates to an apparatus and method for removing noise from a digital photograph, comprising the steps of successively acquiring a peripheral image captured without executing a flash and at least one flash image captured by executing the flash, Applying at least one image alignment technique to the peripheral image and the acquired at least one flash image; acquiring filter weights by applying joint average moving filtering to the acquired at least one flash image; Wherein the step of acquiring the filter weights comprises the steps of: obtaining at least one of the acquired flash image and the wavelet decomposition of the acquired neighboring image by applying the filter weights to the surrounding image, Respectively, and calculating Identifying a valid edge by comparing a subband of the wavelet decomposition of the side image with a subband of wavelet decomposition of the calculated at least one flash image; calculating a noise activity score based on the identified valid edges; Obtaining a size of the kernel for the joint moving average filter using the calculated noise activity score and calculating the filter weights for the sizes of the obtained kernel, The alignment technique may include calculating a transformation between each of the acquired peripheral image and each of the acquired at least one flash image and using each of the acquired at least one of the acquired at least one flash image with the same coordinates .

Description

[0001] APPARATUS AND METHOD FOR NOISE REMOVAL IN A DIGITAL PHOTOGRAPH [0002]

The present invention relates to digital photography, and more particularly, to an apparatus and method for removing noise from a digital photograph.

The resolution of the images that can be captured by the camera of the portable terminal is rapidly improving. However, the mobile form factor imposes restrictions on optical devices that can be incorporated into mobile terminal cameras. That is, the higher the resolution, the higher the complementary metal oxide semiconductor (CMOS) sensor array density, which can result in increased noise in the image under certain illumination / illumination conditions (e.g., in a candlelit environment) have.

Flash is very useful for capturing high-quality photos in low-light conditions. In recent years, it has become possible to integrate an LED flash into a camera of a portable terminal. The range of the LED flash (ie, the effective distance) is sufficient to capture pictures at small meetings, birthday parties, etc., although this is significantly limited compared to conventional flash (xenon flash) equipped with digital cameras or camcorders.

However, the presence of camera flash poses another challenge in digitally capturing precious moments. Camera flash allows you to capture high-quality photos, but disables the effects of the surroundings where the photos are captured. For example, let's consider a birthday party. The surroundings can be a table of birthday cakes on the table with a restaurant table with dingy yellow ambient lighting and candles on top of it. Under such a scenario, the camera of the mobile terminal typically takes a picture of noise and / or fuzzy images. That is, when capturing a picture with an LED flash, the quality of the image may be higher but the surroundings (ie, the original lighting of the environment) are completely compromised.

Until recently, a number of image processing solutions have been used to improve the quality of images to reduce noise effects in the images. Some methods use image data from a single image to estimate filter weights. And applying these weights to filtering the image to reduce noise in the image. However, an edge-stopping term due to the filter is calculated from the image data with noise. If the ambient illumination is low, the estimate may be wrong because the image has overall low contrast, and gradient-based operations may not be very reliable.

Other noise reduction methods typically make assumptions about the statistical properties of the noise affecting the image. For example, the method assumes a correlation between two images or noise in portions of the same image. Thus, the degree of noise reduction appears to depend on the noise correlation of the two images. However, if the correlation coefficient is above a certain threshold, the method may be effective as such, but in that case the assumption can not be applied and the method becomes invalid.

Other noise reduction methods also use various filtering techniques to minimize noise in the image. One of the filters typically used to reduce noise in an image is a bilateral filter. In this case, a single image bidirectional filter is used. The original bidirectional filter is used as an edge-preserving smoothing filter and is similar to a simple single-image mean-shift filter. There is a difference in using local information between the bidirectional filter and the average movement filter. In particular, the bidirectional filter uses a fixed static window, while in the mean-shift filtering, information outside of individual windows is also considered. Therefore, the mean shift filter is superior to the bidirectional filter in edge preservation smoothing. The same limitations of static fixed windows also apply to joint or cross bidirectional filters.

The above methods do not specifically address dual image analysis for noise reduction and image quality improvement. As a result, these methods do not effectively remove noise from the processed image. And, when the flash is turned on and the image is captured, the natural surroundings of the environment are lost. Thus, a need has arisen to provide an effective noise reduction while maintaining the natural characteristics of the environment.

The present invention proposes an apparatus and method for removing noise from a surrounding image by using a flash image.

The present invention proposes an apparatus and method for using a joint average shift filter for noise reduction and edge preservation smoothing of surrounding images.

The present invention also proposes a method and apparatus for selecting a filter bandwidth parameter to obtain effective noise cancellation.

According to an aspect of the present invention, there is provided a method of removing noise from a digital photograph, comprising: obtaining a surrounding image captured without executing a flash and at least one flash image captured by executing the flash continuously Applying at least one image alignment technique to the acquired peripheral image and the acquired at least one flash image; and applying joint averaging filtering to the acquired at least one flash image to obtain filter weights Wherein the step of acquiring the filter weights comprises the steps of: obtaining the filter weights by applying the filter weights to the obtained peripheral images; and acquiring the filter weights by using the obtained at least one flash image, Calculate the wavelet decomposition for the surrounding image, respectively Identifying a valid edge by comparing subbands of wavelet decomposition of the calculated peripheral image with wavelet decomposition subbands of the computed at least one flash image to identify valid edges based on the identified valid edges; Obtaining a size of a kernel for a joint moving average filter using the calculated noise activity score, and calculating the filter weights for sizes of the obtained kernel, Wherein the at least one image alignment technique comprises: calculating a conversion between each of the acquired peripheral image and each of the acquired at least one flash image; and using each of the acquired at least one flash image, To the same coordinates as the surrounding image.
According to another aspect of the present invention, there is provided an apparatus for removing noise from a digital photograph, the apparatus comprising: a peripheral image captured without executing a flash; and at least one flash image captured by executing the flash, Applying at least one image alignment technique to the acquired peripheral image and the acquired at least one flash image and applying joint average movement filtering to the acquired at least one flash image to obtain filter weights And an image processing unit for applying the filter weights to the peripheral image to remove noise of the peripheral image, wherein the image processing unit performs a wavelet decomposition for the obtained at least one flash image and the acquired peripheral image, A wavelet decomposition unit for respectively calculating An edge identification unit for comparing the subband of the wavelet decomposition of the calculated peripheral image with the subband of wavelet decomposition of the calculated at least one of the flash images to identify the valid edges and a noise activity score based on the identified valid edges A kernel estimator for obtaining kernel sizes for the joint moving average filter using the calculated noise activity scores; and a filter for calculating the filter weights for the sizes of the obtained kernels, Wherein the at least one image alignment technique comprises: calculating a conversion between each of the obtained acquired ambient image and each of the acquired at least one flash image; and using the calculated conversion to generate the acquired at least one flash image To the same coordinates as the acquired peripheral image Can.

The present invention can effectively remove noise in the image while maintaining the natural characteristics of the environment.

1 is a block diagram illustrating a method of removing noise according to an embodiment of the present invention;
2 is a flow chart for removing noise according to an embodiment of the present invention;
3 is a flow chart illustrating a filtering mechanism using a joint-averaged moving filter according to an embodiment of the present invention;
4 is a block diagram illustrating a filtering mechanism according to an embodiment of the present invention;
5 illustrates snapshots of filtered images according to an embodiment of the present invention,
Figure 6 is a snapshot of filtered images according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

The present invention provides an apparatus and method for maintaining the original characteristics of an ambient image while providing the clarity and high quality of the captured image using flash. BRIEF DESCRIPTION OF THE DRAWINGS In the drawings, wherein like reference numerals designate like features throughout, and in which: FIG.

In the present invention, an image captured without a flash is referred to as a flash image or a peripheral image.

An apparatus and method for processing a digital image to remove noise from the digital image is disclosed. This method uses a dual image analysis technique. Here, the dual image analysis technique uses a pair of images including an image referred to as a peripheral image photographed in a natural environment and an image that can be photographed with the flash turned on. The above pair of images are taken consecutively in the same environment.

The present invention further utilizes image processing and filtering algorithms. More specifically, the present invention takes a peripheral image as a reference image. And the present invention aligns the flash image with respect to the surrounding image. The present invention also applies a filtering technique to a flash image by using a joint mean shift filter. The present invention also calculates filter weights for the flash image to obtain various functions necessary for algorithm processing. The present invention also applies the calculated filter weights to the surrounding image to obtain an image having the perimeter of the environment and the quality of the flash image.

1 is a block diagram illustrating a method for removing noise according to an embodiment of the present invention.

As shown in Figure 1, the present invention utilizes a dual image analysis technique to remove noise from images. The surrounding image 101 is an image captured without flash under natural environmental conditions. And the flash image 102 is at least one or more images captured by operating the flash in the same environment as the peripheral image 101. [ The at least one or more images are captured continuously with the surrounding image. The surrounding image 101 has a lot of noise, and therefore the image looks blurred and unclear. The flash image 102 is of high quality, but the natural surroundings of the image are completely corrupted.

In view of this, the present invention uses the flash image 102 to obtain filter weights for the surrounding image, and removes noise from the surrounding image 101 using the obtained weights.

More specifically, the portable terminal outputs the flash image and the non-flash (peripheral) image to the image processing technique 103. The portable terminal aligns the images during image processing (103). At this time, the portable terminal can use various image sorting techniques for the images.

After image alignment, the mobile terminal filters the images using an image filtering technique. For example, the handheld terminal uses an algorithm for image filtering 104. In this case, the portable terminal calculates filter weights for the flash image, and applies the calculated weights to the surrounding image to remove noise from the non-flash image. As such, the present invention utilizes a flash image to obtain noise-free peripheral images by removing noise from the non-flash image.

Here, the camera unit of the portable terminal captures the surrounding image and the flash image, the image processing unit obtains the filter weights for the surrounding image using the flash image 102, Can be removed.

2 is a flow chart for removing noise according to an embodiment of the present invention.

The mobile terminal uses a dual image analysis technique using a flash image and a non-flash image. At this time, the portable terminal acquires the filter parameters using the flash image. The mobile terminal then uses algorithms in the non-flash image and applies the obtained parameters to remove noise. In one embodiment, a flash image is used to calculate filter parameters because the flash image has a significantly higher signal-to-noise ratio (SNR) than the surrounding image.

First, the portable terminal captures a surrounding image (without flash) of a desired scene (201). Here, the surrounding image includes the surrounding environment, but is blurred and lacks clarity due to the noise existing in the surrounding image. The portable terminal turns on the flash and captures a pair of flash images of the same scene (202). Here, the flash image does not include a natural surrounding environment.

The mobile terminal aligns the flash image with the non-flash image using an image alignment technique (203). At this time, the image alignment technique may include a scale invariant feature transform (SIFT) and the like. The mobile terminal may then perform the alignment procedure using the candidate set of points calculated by the interest-point detector in the ambient illumination image and the one or more flash illumination images. For example, the detector may include a Harris corner detector, a SIFT detector, and the like.

Here, the alignment procedure improves the candidate set generated by eliminating non-matching point pairs using gradient orientation. The alignment procedure then uses the set of enhanced point pairs to calculate the transformation between the surrounding image and each of the flash images and transforms each of the flash images into the same coordinate system as the surrounding image using the calculated transformations, (204). The portable terminal then outputs sorted images (both images of the flash image and the non-flash image) by filtering.

The mobile terminal utilizes various mean shift filter algorithms. During filtering, the mobile terminal computes (205) the filter weights and kernel functions necessary for the average mobile filtering process using the aligned flash image. The mobile terminal then applies the calculated filter weights to the non-flash image so that the filter removes noise from the non-flash image. In addition, the portable terminal performs a detail transfer to acquire a specific image (206). Where the detail movement ensures that the visible detail is moved from the flash image visible in the flash image to the surrounding image. The mobile terminal obtains a noise-free image in which the sharpness of the flash image and the natural surroundings of the non-flash image are corrected by filtering the specific image.

The various operations of the noise reduction method 200 described above may be performed in the order presented, in a different order, or simultaneously. Further, in some embodiments, some of the operations listed in Fig. 2 may be omitted.

The present invention describes a method for reducing noise in a surrounding image using data from a flash image. To this end, the present invention proposes to use a Joint Average Moving Filter. Here, the joint average shift filter is constructed based on an average shift filtering procedure for a single image. The joint-averaging filter uses a joint spatial-range domain. The spatial domain is composed of lattice grid sites. The range domain is L ^{* It} consists of component data values.

를 갖는 특징 공간의)을 위치시키는 방법을 제공한다. 가중치 계산 절차에서는 그래디언트의 제로들

사이에 모드(mode)들을 위치시킨다. 주어진 데이터 포인트들의 집합에 대한 밀도 추정량(density estimator)은 아래와 같이 나타낼 수 있다.The weight calculation procedure using the average moving filter is as follows. The average moving procedure is based on the assumption that zeroes (underlying density)

Of the feature space). In the weight calculation procedure,

To place the modes between them. The density estimator for a given set of data points can be expressed as:

는 커널 프로파일을 나타내고,

는 대역폭 파라미터를 나타낸다. 이 경우, 모드들을 추정하는 문제는 그래디언트 밀도를 추정하는 문제로 해석된다. 또한, 그래디언트 밀도는 수학식 2와 같이 나타낼 수 있다.In Equation (1)

Represents the kernel profile,

Represents the bandwidth parameter. In this case, the problem of estimating the modes is interpreted as a problem of estimating the gradient density. Further, the gradient density can be expressed by the following equation (2).

로 계산된 x에서의 밀도 추정량에 비례한다. 또한, 커널 G로 계산된 x에서의 밀도 추정량은 다음과 같다.In Equation 2, the first term is the kernel

Is proportional to the density estimate at x computed as. The density estimator at x calculated by kernel G is as follows.

The second term is the average moving term.

를 사용한 가중 평균과 커널(윈도)의 중심 x 간의 차이다. 수학식 2는 수학식 3과 4를 이용하면, 수학식 5와 같이 유도된다. That is, the average moving term is the kernel for the weights

And the center of the kernel (window) x. Equation (2) is derived as Equation (5) using equations (3) and (4).

From Equation (5), the following Equation (6) is derived.

로 계산된 평균 이동 벡터가 커널

로 얻은 정규화된 밀도 그래디언트 추정량에 비례한다는 것을 보여준다. 정규화는 커널

로 계산된 x에서의 밀도 추정량에 의해 된다. 따라서 평균 이동 벡터는 항상 밀도의 최대 증가의 방향 쪽을 향하게 된다.Equation (6)

≪ / RTI >< RTI ID = 0.0 >

Is proportional to the normalized density gradient estimator obtained with < RTI ID = 0.0 > Normalization is the kernel

Lt; RTI ID = 0.0 > x. ≪ / RTI > Therefore, the average motion vector always faces the direction of the maximum increase in density.

The above-mentioned procedure utilizes a joint averaging shift filter and seeks all the advantages of a normal averaging shift filter procedure by a typical bidirectional filter procedure.

A feature of the present invention lies in calculating filter weights from a flash image instead of a non-flash image. It is a feature of the present invention to improve the computational efficiency of the method by replacing the iterative method of the mean moving filter with a single iterative step.

를 계산하는 것은 비플래시 이미지에서 동일 커널을 계산하는 것보다 더 신뢰성 있는 추정량을 산출한다.The above method computes the filter weights using a flash image, resulting in better noise reduction than using only non-flash images. This can be explained by the fact that the flash image has a higher signal-to-noise ratio than the surrounding image. Kernel from flash image

Computes a more reliable estimator than computing the same kernel in a non-flash image.

In addition, any suitable kernel that meets the conditions for a valid kernel can be used to implement the proposed method. In certain embodiments, using a uniform kernel has proved satisfactory. A uniform kernel appears to guarantee convergence in a finite number of steps. This is an important consideration in the practical implementation of the method. Of course, it should be noted that not only a uniform kernel but also other kernel profiles can be used.

In another particular embodiment, using a normal kernel, the convergence rate by the normal profile is slower, although the result is almost always significantly better.

를 변경/선택함으로써 필터링(평할화 또는 노이즈 제거)의 정도를 설정할 수 있다는 것이다. 이것은 사용자 또는 조작자가 다수의 복잡한 파라미터들을 선택할 필요가 없기 때문에 상당한 이점이 된다.Another important aspect in implementing this method is that the filter bandwidth parameter, a single parameter

(Leveling or noise reduction) can be set by changing / selecting the threshold value. This is a significant advantage because the user or operator does not need to select a large number of complex parameters.

를 선택하는 것도 제안하고 있다. 본 발명은 주변 이미지의 노이즈 추정량을 계산하여 적절한 대역폭 파라미터를 선택할 수 있다. 이제부터, 도 3을 이용하여 대역폭 파라미터를 선택하는 것에 대하여 설명하고자 한다.
The present invention is based on the analysis of the surrounding image,

As shown in FIG. The present invention can select the appropriate bandwidth parameters by calculating the noise estimate of the surrounding image. The selection of the bandwidth parameter will now be described with reference to FIG.

를 선택하는 것을 지원한다. 그리고 본 발명은 주변 이미지의 노이즈 추정량을 계산하여 적절한 대역폭 파라미터를 선택할 수 있다.3 is a flow diagram illustrating a filtering mechanism using a joint-averaging filter in accordance with an embodiment of the present invention. The present invention is based on the analysis of the surrounding image,

Is supported. The present invention can then calculate the noise estimate of the surrounding image to select the appropriate bandwidth parameter.

The mobile terminal calculates wavelet decompositions separately for the surrounding image and the flash image (301). At this time, the portable terminal performs a single wavelet decomposition. The mobile terminal compares the HH subbands of the flash wavelet decomposition and the surrounding wavelet decomposition to identify valid edges (302). The portable terminal calculates an activity score for the surrounding image (303). Here, the activity score is an index of the noise level in the surrounding image. At this time, the mobile terminal calculates HH subband and edge information calculated in step 302.

More specifically, the wireless terminal thresholds the wavelet coefficients for a specified threshold and calculates the fraction of coefficients over the threshold for the total number of wavelet coefficients in a pre-given region of the HH subband (a predetermined window) And calculates the activity score. Here, the threshold can be set by analyzing the difference between the flash image and the surrounding image or one of the HH subbands thereof. Then, the mobile terminal calculates the activity score calculated for the window for windows that do not overlap across the entire image. The mobile terminal then calculates the activity score for the surrounding image as the average of the activity scores for all windows examined.

The mobile terminal then selects among the predetermined kernel sizes for the average movement filter using the noise activity score (304), and the predetermined kernel sizes are low and high, corresponding to the degree of noise in the original peripheral image high, and midium (or weak, average, and strong). The portable terminal converts the flash image and the surrounding image into L ^* a ^* b ^* color space (305).

The mobile terminal calculates the weights for a given kernel using the flash image (306). At this time, the mobile terminal can calculate the filter weights using the kernel function, where the independent variables for the kernel function are the pixel values of the flash image instead of the pixel values of the filtered image. The portable terminal filters the surrounding images using the calculated kernel weights and removes noise from the surrounding images (307). The mobile terminal then computes kernel function values across the pixel being filtered or the neighbor of the input pixel (in the joint space-range domain). The portable terminal converts the surrounding image into a required format (308). Such a format may be a conversion from L ^* a ^* b ^* to RGB, and the like.

The operations shown in Fig. 3 may be performed in the order presented, in a different order, or simultaneously. Further, in some embodiments, some of the operations shown in Fig. 3 may be omitted.

4 is a block diagram illustrating a filtering mechanism according to an embodiment of the present invention.

Referring to FIG. 4, the flash image acquiring unit 101 photographs a flash image, and outputs the photographed flash image to the wavelet decomposition unit 401a and the L ^* a ^* b ^* conversion unit 405a. The surrounding image obtaining unit 102 photographs the surrounding image and outputs the captured surrounding image to the wavelet decomposition unit 401b and the L ^* a ^* b ^* conversion unit 405b. Then, the wavelet decomposition unit 401a obtains a wavelet for the flash image by decomposing the input flash image using a predetermined wavelet decomposition method. Then, the wavelet decomposition unit 401b decomposes the inputted surrounding image using a predetermined wavelet decomposition method to obtain a wavelet for the surrounding image.

The edge identification unit 402 then compares the HH subbands between the flash image and the surrounding image to identify the valid edges. Here, the edges provide an indication of the noise level for the surrounding image. The noise activity calculation unit 403 calculates the noise activity using the HH subband and edge information of the calculated non-flash image (surrounding image). Here, the edge information includes information on valid edges.

Then, the kernel estimating unit 404 selects one of the predetermined kernel sizes (low, high, and medium) for the average moving filter using the score of the noise activity (kernel estimation). And L ^* a ^* b ^* conversion unit (405a) is a flash image, the L ^* a ^* b ^* conversion into a color space, and L ^* a ^* b ^* conversion unit (405b) has a peripheral image of L ^* a ^* b ^* color Space. For example, when the surrounding image and the flash image exist in the RGB color space, the L ^* a ^* b ^* conversion units 405a and 405b convert the surrounding image and the flash image into the L ^* a ^* b ^* color space .

The filter 406 filters the surrounding image using the weight of the estimated kernel size. At this time, the filter can calculate the weights necessary for the filter used in the joint moving average filter using the above-described algorithms and mathematical equations. Thus, by filtering the surrounding images using the computed kernel weights, the final noisy surrounding image is obtained. The RGB conversion unit 407 converts the final peripheral image into RGB color space.

Through the above operation, the present invention can provide a single iteration of the mean shifting filter to calculate the filtered (noise reduced) pixel value of the non-flash image.

And the present invention can appropriately perform the joint average shift filter procedure. That is, the present invention may replace the ambient illumination image pixel data with the filtered pixel data so that there is no degradation in the final image.

를 적응적으로 선택하는 자동화된 방법을 사용할 수 있다.
And the present invention may be implemented by a computer program that executes the above-described method in an application. In this case, such a computer program only uses integer arithmetic for all operations. In addition, such a computer program may be used to generate a bandwidth parameter vector < RTI ID = 0.0 >

Lt; RTI ID = 0.0 > adaptive < / RTI >

Figure 5 is a snapshot of filtered images according to an embodiment of the present invention.

501 screen indicates that the surrounding environment of the flash image is lacking. The 503 screen shows the surrounding image taken under low light conditions, and the 505 screen shows the filtered image. Here, the filtered image is filtered using the above-mentioned algorithms. It can be seen that the filtered image has less noise and maintains the surrounding environment of the original image.

Figure 6 is a snapshot of filtered images according to an embodiment of the present invention.

601 screen indicates that the surrounding environment of the flash image is lacking. The 603 screen shows the surrounding image taken under low light conditions, and the 605 screen shows the filtered image. Here, the filtered image is filtered using the above-mentioned algorithms. It can be seen that the filtered image has less noise and maintains the surrounding environment of the original image.

The embodiments disclosed herein may be implemented through at least one software that performs network management functions that are executed on at least one hardware device and control elements. The elements shown in Figures 1 and 4 include hardware devices and blocks that may be at least one of a combination of a hardware device and a software module.

Although the embodiments of the present invention have been described in connection with the mobile terminal, various modifications may be made without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited by the described embodiments but should be determined by the equivalents of the claims and the claims.

101: flash image acquisition unit 102: peripheral image acquisition unit
401a, 401b: wavelet decomposition section 402: edge identification section
403: Noise activity calculation unit 404: Kernel estimation unit
405a, 405b: L ^* a ^* b ^* conversion unit 406: filter
407: RGB conversion unit

Claims (20)

A method for removing noise from a digital photograph,
The method comprising the steps of: continuously acquiring a peripheral image captured without executing a flash and at least one flash image captured by executing the flash;
Applying at least one image alignment technique to the acquired peripheral image and the acquired at least one flash image;
Acquiring filter weights by applying joint moving average filtering to the obtained at least one flash image;
And removing the noise of the surrounding image by applying the filter weights to the obtained surrounding image,
The process of acquiring the filter weights comprises:
Calculating wavelet decomposition of the obtained at least one flash image and the obtained surrounding image, respectively;
Identifying valid edges by comparing subbands of wavelet decomposition of the calculated peripheral image with subbands of wavelet decomposition of the calculated at least one flash image;
Calculating a noise activity score based on the identified valid edges;
Acquiring kernel sizes for the joint moving average filter using the calculated noise activity scores;
And calculating the filter weights for the sizes of the obtained kernel,
Wherein the at least one image alignment technique further comprises: calculating a conversion between each of the obtained acquired ambient image and each of the acquired at least one flash image; and using each of the acquired at least one flash image using the calculated conversion, A method of noise reduction that transforms to the same coordinates as the surrounding image.
The method according to claim 1,
Wherein the at least one image alignment technique uses a set of candidate points comprising the acquired ambient image and points calculated by an interest point detector from the acquired at least one flash image.
3. The method of claim 2,
Wherein the point of interest detector comprises at least one of a Harris corner detector and a scale invariant feature transform (SIFT) detector.
3. The method of claim 2,
Wherein the step of applying the at least one image-
And improving the candidate point set by removing unmatched pairs of points among the points included in the candidate point set using a gradient orientation.
The method according to claim 1,
Wherein the step of applying the joint moving average filtering comprises:
And converting the color image data from the acquired peripheral image and the acquired at least one flash image into a uniform color space,
Wherein the uniform color space includes at least one of an L ^* u ^* v ^* color space and an L ^* a ^* b ^* color space.
The method according to claim 1,
The joint mean-shift filtering includes a joint space-range domain,
Among the joint space-range domains, the spatial domain includes grid grid sites,
Wherein the range domain of the joint space-range domain comprises L ^* component data values.
The method according to claim 1,
The joint moving average filtering may be performed by:
And a single iterative averaging filter for filtering the obtained surrounding image.
The method according to claim 1,
And selecting a degree of filtering by selecting a bandwidth parameter for the joint moving average filter.
delete The method according to claim 1,
The process of calculating the wavelet decomposition, respectively,
And calculating single wavelet decomposition for the obtained at least one flash image and the obtained surrounding image.
An apparatus for removing noise from a digital photograph,
A camera unit for successively acquiring a peripheral image captured without executing a flash and at least one flash image captured by executing the flash;
Applying at least one image alignment technique to the acquired peripheral image and the acquired at least one flash image, applying joint averaging motion filtering to the acquired at least one flash image to obtain filter weights, And an image processor for removing the noise of the surrounding image by applying the filter weights,
Wherein the image processing unit comprises:
A wavelet decomposition unit for respectively calculating the obtained at least one flash image and wavelet decomposition for the obtained surrounding image,
An edge identification unit for comparing the calculated subband of wavelet decomposition of the surrounding image with the subband of wavelet decomposition of the calculated at least one flash image to identify valid edges;
A noise activity calculator for calculating a noise activity score based on the identified valid edges;
A kernel estimator for obtaining sizes of kernels for the joint moving average filter using the calculated noise activity scores;
And a filter for calculating the filter weights for the sizes of the obtained kernel,
Wherein the at least one image alignment technique further comprises: calculating a conversion between each of the obtained acquired ambient image and each of the acquired at least one flash image; and using each of the acquired at least one flash image using the calculated conversion, To the same coordinates as the surrounding image.
12. The method of claim 11,
Wherein the image processing unit comprises:
And using a set of candidate points including the acquired peripheral image and the points calculated by the point of interest detector from the acquired at least one flash image.
13. The method of claim 12,
Wherein the point of interest detector comprises:
A Harris corner detector and a SIFT detector.
13. The method of claim 12,
Wherein the image processing unit comprises:
And corrects the candidate point set by removing unmatched pairs of points among the points included in the candidate point set using a gradient orientation.
12. The method of claim 11,
Wherein the image processing unit comprises:
Wherein the image processing unit comprises:
Convert the color image data from the acquired peripheral image and the acquired at least one flash image into a uniform color space,
Wherein the uniform color space includes at least one of an L ^* u ^* v ^* color space and an L ^* a ^* b ^* color space.
12. The method of claim 11,
The joint mean-shift filtering includes a joint space-range domain,
Among the joint space-range domains, the spatial domain includes grid grid sites,
Wherein the range domain of the joint space-range domain comprises L ^* component data values.
12. The method of claim 11,
Wherein the joint moving average filtering includes a single iterative moving average filter for filtering the obtained surrounding image.
12. The method of claim 11,
Wherein the image processing unit selects a degree of filtering by selecting a bandwidth parameter for the joint moving average filter.
delete 12. The method of claim 11,
Wherein the wavelet decomposition unit comprises:
And calculate single wavelet decomposition for the obtained at least one flash image and the obtained surrounding image.

Images