KR20100070053A - Apparatus for processing digital image and method for controlling thereof - Google Patents

Abstract

PURPOSE: An apparatus for processing a digital image and a method for controlling thereof are provided to perform out-focusing effect around a subject by detecting the movement of the subject in a moving picture photographing process. CONSTITUTION: An image input unit(310) receives an input image. A controller(320) detects a moving subject from the input image and extracts an area which the moving subject is occupied as a subject area. The controller performs a sharpening filtering process with respect to the substrate area and performs a blurring filtering process with respect to an area excluding the substrate area. A storing unit(340) stores the sharpening and blurring filtered image.

Description

Apparatus for processing digital image and method for controlling Technical Field

The present invention relates to a digital image processing apparatus and a control method thereof, and more particularly, to a digital image processing apparatus and a method of controlling the same, wherein a moving picture photographing function of continuously photographing a subject's movement is implemented.

Typically, the digital image processing apparatus includes all devices that process images such as a digital camera, a personal digital assistant (PDA), a phone camera, a PC camera, or use an image recognition sensor.

The digital image processing apparatus may image an image received through an image pickup device by a digital signal processor, compress the image, generate an image file, and store the image file in a memory.

In addition, the digital image processing apparatus may display and display an image of an image file received through an imaging device or stored in a storage medium on a display device such as a liquid crystal display (LCD).

In general, the digital image processing apparatus may include a moving picture photographing function of continuously photographing a moving object. In this case, the image data input to the image sensor through the lens is stored in a set file format by performing general image processing without additional processing.

An object of the present invention is to provide a digital image processing apparatus and a method of controlling the same, capable of realizing an out-focusing effect around a subject by detecting a movement of the subject during video shooting.

The present invention, the step of receiving an input image; Extracting a subject area from the input image; Performing sharpening filter processing on the subject area in the input image; And performing a blurring filter process on the region other than the subject region in the input image.

The method may further include detecting a moving subject from the input image.

The detecting of the moving subject may include calculating a motion vector by comparing the input image with an image of a previously input frame, and tracking the moving subject from the motion vector.

An area occupied by the moving subject may be extracted as the subject region.

The input image may be an image of one frame of an input video in which images of a plurality of frames are consecutive.

The method may further include storing the converted image generated by the sharpening filter process and the blurring filter process on the input image.

Another aspect of the invention, the image input unit for receiving an input image; And a controller configured to extract a subject region from the input image, perform a sharpening filter process on the subject region in the input image, and perform a blurring filter process on the region other than the subject region in the input image. Provided is an image processing apparatus.

The moving subject may be detected from the input image, and an area occupied by the moving subject may be extracted as the subject region.

A motion vector may be calculated by comparing the input image with an image of a frame previously input, and the area occupied by the moving subject may be extracted as the subject area by tracking the moving subject from the motion vector.

The input image may be an image of one frame of an input video in which images of a plurality of frames are consecutive.

The storage apparatus may further include a storage unit configured to store the converted image generated by the sharpening filter process and the blurring filter process on the input image.

According to the digital image processing apparatus and the control method thereof according to the present invention, an out-focusing effect can be realized centering on a subject by detecting a movement of the subject during video shooting.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a rear view of a digital camera 100 which is an embodiment of a digital image processing apparatus according to the present invention.

Referring to the drawings, on the back of the digital camera 100, the direction button 21, the menu-OK button 22, the wide angle (Zoom) button (W), the telephoto (Zoom) ) Button T, the display panel 25 and the like.

The direction button 21 may include four buttons, an up button 21a, a down button 21b, a left button 21c, and a right button 21d. The direction button 21 and the menu-OK button 22 are keys for executing various menus relating to the operation of the digital image processing apparatus such as a digital camera.

The wide-angle zoom button W or the tele-zoom button T has a wider angle of view or a narrower angle of view depending on its input. In particular, it may be used to change the size of the selected exposure area. In this case, when the wide-zoom button W is input, the size of the selected exposure area increases, and when the tele-zoom button T is input, the size of the selected exposure area may decrease.

As the display panel 25, an image display device such as a liquid crystal display (LCD) may be used. The display panel 25 may be included in the display unit 350 of FIG. 3, which displays an input video or a stored video.

Meanwhile, a shutter release button 26, a flash (not shown), a power switch 28, and a lens unit (not shown) may be provided on the front or top surface of the digital camera 100. In addition, the front and rear of the digital camera 100 may be provided with an objective lens and an eyepiece of the viewfinder 27.

The shutter release button 26 and the power switch 28 may be included in a user manipulation unit (360 of FIG. 3) for inputting items to be manipulated by the user from the outside.

The shutter release button 26 is opened and closed to expose the film or an image pickup device such as a charge coupled device (CCD) for a predetermined time. In addition, the shutter release button records an image on the image pickup device by properly exposing a subject in association with an aperture (not shown).

On the other hand, the shutter release button 26 can be pressed in two stages. When the shutter release button 26 is pressed in one end, the first signal S1 may be generated, and when the shutter release button 26 is pressed in one step, the second signal S2 may be generated. The shooting preparation setting may be performed by the first signal S1, and the shooting operation may be performed by the second signal S2.

As an embodiment of a digital image processing apparatus to which the present invention can be applied, a digital camera, a control apparatus, and a control method thereof are disclosed in US Patent Application Publication No. 2004/0130650 (name: automatic using a secondary function of the camera). Method of automatically focusing using a quadratic function in camera.

Matters relating to the digital camera, the control apparatus, and the control method disclosed in the above-mentioned US application will be included in the present specification, the detailed description thereof will be omitted.

2 is a block diagram of a control device 200 of a digital image processing apparatus, which is a preferred embodiment of the present invention. The control device 200 of the digital image processing apparatus may be mounted in the digital camera 100 of FIG. 1.

Referring to the drawing, an optical system OPS including a lens unit and a filter unit optically processes light from a subject. The lens unit of the optical system OPS includes a zoom lens, a focus lens, and a compensation lens. When the user presses the wide-angle zoom button W or the tele-zoom button T included in the user input unit INP, a signal corresponding thereto is input to the microcontroller 212.

Accordingly, as the microcontroller 212 controls the lens driver 210, the zoom motor M _Z is driven to move the zoom lens. That is, when the wide-zoom button W is pressed, the focal length of the zoom lens is shortened to widen the angle of view, and when the tele-zoom button T is pressed, the focal length of the zoom lens becomes long and the angle of view is narrowed.

On the other hand, in the auto focusing mode, the main controller embedded in the digital signal processor 207 controls the lens driver 210 through the microcontroller 212, and thus the focus motor M _F is driven. do. That is, the focus lens is moved to a position where the sharpest picture is obtained by driving the focus motor M _F.

The auto focus mode may be performed according to the first signal S1 generated by pressing the shutter release button 26 of FIG. 1 input through the user input unit INP. At this time, the plurality of images are input while the position of the focus lens is moved along the optical axis, and the position of the focus lens which obtains the clearest image among the plurality of images is obtained.

The compensation lens compensates for the overall refractive index and is not driven separately. Reference numeral M _A denotes an aperture adjusting motor M _A for driving an aperture (not shown).

In the filter section of the optical system OPS, an optical low pass filter removes optical noise of high frequency components. Infra-Red cut filter cuts the infrared component of the incident light.

The photoelectric conversion unit OEC may include an imaging device such as a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS). The photoelectric conversion unit OEC converts light from the optical system OPS into an electrical analog signal.

The analog-to-digital converter may include a CDS-ADC (Correlation Double Sampler and Analog-to-Digital Converter) device 201. The analog-to-digital converter processes the analog signal from the photoelectric converter (OEC), removes the high frequency noise, adjusts the amplitude, and converts the analog signal into a digital signal. Here, the digital signal processor 207 controls the timing circuit 202 to control the operation of the photoelectric converter OEC and the analog-to-digital converter 201.

The optical system OPS, the photoelectric conversion unit OEC, the CDS-ADC element 201, and the like may be included in the image input unit 310 of FIG. 3 to which an input image is input according to the present invention.

The real time clock 203 provides time information to the digital signal processor 207. The digital signal processor 207 processes digital signals from the CDS-ADC element 201 to generate digital image signals classified into luminance (Y value) and chromaticity (R, G, B) signals.

The light emitting unit LAMP driven by the microcontroller 212 under the control of the main controller built in the digital signal processor 207 includes a self-timer lamp, an auto-focus lamp, a mode indicating lamp, a flash standby lamp, and the like. May be included.

The user input unit INP may include a direction button 21, a wide-angle zoom button W, a telephoto-zoom button T, and the like. The user input unit INP may be included in the user manipulation unit 360 of FIG. 3.

The digital image signal from the digital signal processor 207 is temporarily stored in the DRAM (Dynamic Random Access Memory) 204. The EEPROM (Electrically Erasable and Programmable Read Only Memory) 205 stores algorithms and setting data such as a boot program and a key input program required for the operation of the digital signal processor 207. In the memory card interface 206, a user's memory card may be attached or detached.

The digital signal processor 207 and / or the microcontroller 212 may be included in the controller 320 of FIG. 3 according to the present invention. The digital signal processor 207 and / or the microcontroller 212 may also be equipped with cache memory as temporary storage space.

In this case, the cache memory and the DRAM 204 may be included in the first storage unit 330 of FIG. 3 which temporarily stores the previous image and the input image. The cache memory included in the first storage unit 330 of FIG. 3 may be included separately from the digital signal processor 207 and / or the micro controller 212.

The memory card recognized through the memory card interface 206 may be included in the second storage unit 340 of FIG. 3, in which a moving image including a converted image generated according to the present invention is non-volatilely stored.

The digital image signal from the digital signal processor 207 is input to the display panel driver 214, whereby the image is displayed on the display panel 215.

The controller 200 of the digital image processing apparatus may further include display units 214 and 215. The display units 214 and 215 may include a display panel 215 and a display panel driver 214 for driving the display panel 215. The display units 214 and 215 may be included in the display unit 350 of FIG. 3 according to the present invention.

On the other hand, the digital image signal from the digital signal processor 207 can be transmitted by serial communication via a universal serial bus (USB) connection portion 31a or an RS232C interface 208 and the connection portion 31b thereof, and a video filter ( 209 and through the video output unit 31c. Here, the digital signal processor 207 may incorporate a microcontroller therein.

The audio processor 213 outputs the audio signal from the microphone MIC to the digital signal processor 207 or the speaker SP, and outputs the audio signal from the digital signal processor 207 to the speaker SP.

3 is a block diagram of a digital image processing apparatus 300 according to a preferred embodiment of the present invention. The digital image processing apparatus 300 may be controlled by the control method S400 of the digital image processing apparatus illustrated in FIG. 4. For the same matters as those described for the control method (S400) of the digital image processing apparatus 300 in the digital image processing apparatus 300, this description will be omitted.

Referring to the drawings, the digital image processing apparatus 300 includes an image input unit 310; The control unit 320; Storage units 330 and 340; Display unit 350; And a user manipulation unit 360.

The image input unit 310 receives an input image. The controller 320 extracts a subject region from the input image, performs a sharpening filter process on the subject region in the input image, and performs a blurring filter process on the region other than the subject region in the input image.

The digital image processing apparatus 300 according to the present invention detects a movement of a subject from an input image corresponding to each frame by receiving a video input, extracts a moving subject region, and performs a sharpening filter process on the subject region. A blurring filter process can be performed on areas other than the area.

Accordingly, the moving subject region is clearly seen, and other regions are blurred, so that an out focusing effect can be obtained with respect to the moving subject with respect to the moving subject region.

The input image received through the image input unit 310 may be an image of one frame of a moving image in which images of a plurality of frames are consecutively input.

The controller 320 may detect a moving subject in the input image and extract a region occupied by the moving subject as the subject region. In this case, a motion vector may be calculated by comparing an input image with an image of a frame previously input, and a moving subject may be tracked from the motion vector to extract a region occupied by the moving subject as a subject region.

The moving subject can be detected by motion tracking. Motion tracking calculates a motion vector by determining a difference between two consecutive frame images, and tracks a subject by the motion vector.

In this case, the motion vector may be obtained by setting a feature point in the currently input image and calculating how much data of the feature point coordinates moves in comparison with the previous image of the previously input frame. In this case, the feature point may be set by the user and may be automatically set by comparison with a previous image.

A sharpening filter process may be performed on the subject area in the entire input image. In addition, a blurring filter process may be performed on the entire area of the input image except for the subject area.

In this case, it may be determined whether the entire image area belongs to the subject area in units of pixels. In addition, a sharpening filter process or a blurring filter process may be performed for each pixel unit.

The sharpening filter and the blurring filter can be set in various forms. The sharpening filter and the blurring filter may be set to 3 × 3 size in consideration of the processing time as shown in FIGS. 5 and 6. As another embodiment, larger sizes of filters, such as 5x5 or 7x7, can be used where processing time is not a problem.

5 illustrates an embodiment of a blurring filter 50 used in a control method of a digital image processing apparatus according to the present invention. 6 shows an embodiment of a sharpening filter 60 used in the control method of the digital image processing apparatus according to the present invention.

In addition, the storage units 330 and 340 may store the converted image generated by performing a sharpening filter process and a blurring filter process on the input image.

The display unit 350 may display the input image in live view. In this case, the converted image converted according to the present invention may be displayed in a live view. The image input unit 310 receives an input image from the outside. The image input unit 310 may include an optical system (OPS), a photoelectric converter (OEC), a CDS-ADC device 201, and the like illustrated in FIG. 2.

The controller 320 may include an image input unit 310; The control unit 320; Storage units 330 and 340; Display unit 350; And the user operating unit 360 to control the focusing effect on the moving subject. The controller 320 may include the digital signal processor 207 and / or the microcontroller 212 illustrated in FIG. 2.

The storage unit 330 or 340 may store an image of a previous frame, an input image, and a moving image generated by a converted image generated according to the present invention. The storage units 330 and 340 may include a first storage unit 330 and a second storage unit 340. The first storage unit 330 may temporarily store the image of the previous frame and the input image. The second storage unit 340 may store a moving image captured and formed by the converted image generated according to the present invention.

The display unit 350 may include the display panel 25 shown in FIG. 1 and / or the display panel driver 214 and the display panel 215 of FIG. 2. The user may input a desired command from the outside through the user manipulation unit 360. The user manipulation unit 360 may include a shutter release button 26 and a power switch 28 of FIG. 1 and / or a user input unit INP of FIG. 2.

According to the present invention, an out-focusing effect can be realized based on a moving subject by detecting a movement of the subject during video recording.

4 is a flowchart illustrating a control method (S400) of a digital image processing apparatus as a preferred embodiment according to the present invention.

The control method S400 of the digital image processing apparatus may be implemented in the digital image processing apparatus of FIGS. 2 and / or 3 and the control apparatuses 200 and 300 thereof. To this end, the control method S400 of the digital image processing apparatus according to the present invention may be a program or algorithm stored in the storage means of FIG. 2 or implemented in the form of a semiconductor chip such as firmware.

Therefore, in the method S400 of controlling the digital image processing apparatus, the same matters as those described for the digital image processing apparatus and the control apparatuses 200 and 300 will be referred to, and detailed description thereof will be omitted.

Referring to the drawings, the control method of the digital image processing apparatus (S400), the input image input step (S420); Detecting a moving subject (S430); A subject region extraction step (S440); Sharpening processing step (S460); And a blurring processing step (S470).

In the input image input step (S420), the input image is received. In the detecting of the moving subject (S430), the moving subject is detected from the input image. In the subject region extraction step (S440), a subject region is extracted from the input image.

In the sharpening process step (S460), a sharpening filter process is performed on the subject area in the input image. In the blurring process step S470, a blurring filter process is performed on an area of the input image except for the subject area.

The control method (S400) of the digital image processing apparatus according to the present invention receives a moving image, detects a movement of a subject from an input image corresponding to each frame, extracts a moving subject region, and processes a sharpening filter on the subject region. Then, blurring filter processing can be performed on areas other than the subject area.

The control method (S400) of the digital image processing apparatus according to the present invention may include a video shooting mode determination step (S410). In the video shooting mode determination step (S410), it may be determined whether the current shooting mode is the video shooting mode. In this case, when it is determined that the video recording mode is determined, an out-focusing effect may be generated based on the moving subject area by detecting the moving subject area.

Accordingly, the moving subject region is clearly seen, and other regions are blurred, so that an out focusing effect can be obtained with respect to the moving subject with respect to the moving subject region.

The input image received in the input image input step S420 may be an image of one frame of the input video in which images of a plurality of frames are consecutive.

In the moving subject detecting step S430, the moving subject may be detected by motion tracking. Motion tracking calculates a motion vector by determining a difference k between two consecutive frame images, and tracks a subject by the motion vector.

In this case, the motion vector may be obtained by setting a feature point in the currently input image and calculating how much data of the feature point coordinates moves in comparison with the previous image of the previously input frame. In this case, the feature point may be set by the user and may be automatically set by comparison with a previous image.

When motion tracking is performed on a moving person, after finding a feature point for a person or an object, a motion vector of the feature point is obtained, and when the motion vector is changed by a predetermined value or more, it may be determined that the subject has moved.

To this end, the moving subject detecting step S430 may include a motion vector calculating step and a subject tracking step. In the motion vector calculation step, a motion vector may be calculated by comparing an input image with an image of a previously input frame. In the subject tracking step, a moving subject may be tracked from the motion vector.

In the subject region extraction step (S440), an area occupied by the moving subject may be extracted as the subject region. In this case, the moving subject may be detected by motion tracking in the moving subject detecting step S430.

In the sharpening process step (S460), a sharpening filter process may be performed on the subject area in the entire input image. In the blurring process step S470, a blurring filter process may be performed on the entire area of the input image except for the subject area.

At this time, in the subject region determining step (S450), it may be determined whether the subject region is the entire input image. In this case, the sharpening process step S460 may be performed on the subject area, and the blurring process step S470 may be performed on the area except the subject area.

In the subject region determination step (S450), it may be determined whether the object region belongs to the subject region in units of pixels for the entire input image region. That is, it may be determined whether each pixel belongs to the subject area or does not belong to the subject area.

The sharpening filter and the blurring filter can be set in various forms. The sharpening filter and the blurring filter may be set to 3 × 3 size in consideration of the processing time as shown in FIGS. 5 and 6. As another embodiment, larger sizes of filters, such as 5x5 or 7x7, can be used where processing time is not a problem.

5 illustrates an embodiment of a blurring filter 50 used in a control method of a digital image processing apparatus according to the present invention. 6 illustrates an embodiment of the sharpening filter 60 used in the control method of the digital image processing apparatus according to the present invention.

In the sharpening process step (S460), all pixels that are determined to be moving subject areas may be masked by the sharpening filter 60. In the blurring process S470, the blurring filter 50 may mask all pixels that are not determined to be moving subject areas.

Through the subject region determining step S450, the sharpening process S460, and the blurring process S470, all pixels included in the input image may be processed to generate a converted image.

On the other hand, the control method (S400) of the digital image processing apparatus according to the present invention may further comprise a converted image storage step (S480). In the converted image storing step (S480), the converted image generated by performing a sharpening filter process and a blurring filter process on the input image may be stored.

According to the present invention, an out-focusing effect can be realized based on a moving subject by detecting a movement of the subject during video recording.

Although the present invention has been described with reference to one embodiment shown in the accompanying drawings, it is merely an example, and those skilled in the art may realize various modifications and equivalent other embodiments therefrom. I can understand. Accordingly, the true scope of protection of the invention should be defined only by the appended claims.

1 is a diagram illustrating a rear surface of a digital camera as an embodiment of a digital image processing apparatus according to the present invention.

FIG. 2 is a block diagram schematically illustrating a control apparatus that may be included in the digital camera of FIG. 1.

3 is a block diagram schematically showing a digital image processing apparatus according to a preferred embodiment of the present invention.

4 is a flowchart schematically illustrating a control method of a digital image processing apparatus as a preferred embodiment according to the present invention.

5 is a diagram schematically showing an embodiment of a blurring filter used in a digital image processing apparatus and a control method according to the present invention.

6 is a diagram schematically showing an embodiment of a sharpening filter used in the digital image processing apparatus and the control method according to the present invention.

Claims (11)

Receiving an input image; Extracting a subject area from the input image; Performing sharpening filter processing on the subject area in the input image; And And performing a blurring filter process on the region other than the subject region in the input image. The method of claim 1, And detecting a moving subject from the input image. The method of claim 2, Detecting the moving subject, Calculating a motion vector by comparing the input image with an image of a previously input frame, and And tracking the moving subject from the motion vector. The method of claim 2, The control method of the digital image processing apparatus for extracting the area occupied by the moving subject to the subject area. The method of claim 1, And the input image is an image of one frame of a moving image in which images of a plurality of frames are consecutive. The method of claim 1, And storing the converted image generated by performing the sharpening filter process and the blurring filter process on the input image. An image input unit which receives an input image; And A digital image including a controller which extracts a subject region from the input image, performs a sharpening filter process on the subject region in the input image, and performs a blurring filter process on an area except the subject region in the input image Processing unit. The method of claim 7, wherein And detecting a moving subject from the input image and extracting a region occupied by the moving subject as the subject region. The method of claim 8, And calculating a motion vector by comparing the input image with an image of a frame previously input, and extracting a region occupied by the moving subject into the subject region by tracking the moving subject from the motion vector. The method of claim 7, wherein And the input image is an image of one frame of a moving image in which images of a plurality of frames are consecutive. The method of claim 7, wherein And a storage unit configured to store the converted image generated by the sharpening filter process and the blurring filter process on the input image.

Images