JP2005100144A - Image processing method, device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To avoid a device for picking up digital photo images from enlarging and to deblur the images. <P>SOLUTION: A process in which a correction means 10 actually subjects a blurred digital photo image D to a directional correction process in a plurality of different directions Ri to obtain a plurality of processed images Di matching the respective directions, a process in which an evaluation value calculation means 20 evaluates the degree of blurring in each processed image Di, a process in which a direction selecting means 30 selects the direction Rj matching the processed image least blurred, and a process for resetting near the direction Rj a direction Ri in which to perform the directional correction process are repeated until predetermined requirements are met. The processed image matching the direction finally selected is acquired as a deblurred digital photo image D'. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing method and apparatus for correcting camera shake in a digital photographic image, and a program therefor.

  Digital photographic images obtained by photoelectrically reading photographic images recorded on photographic films such as negative films and color reversal films with a reading device such as a scanner, and digital photographic images obtained by taking images with a digital still camera (DSC) For example, printing with various image processing is performed. As one of these image processes, there is a blurred image restoration process that removes a blur from a blurred image (blurred image).

  Reasons for blurring a photographic image obtained by capturing an image of a subject include out-of-focus due to inconsistent focal length and blur due to camera shake of the photographer (hereinafter referred to as camera shake). In the case of out-of-focus, the point image spreads two-dimensionally, that is, the spread on the photographic image exhibits non-directionality, whereas in the case of camera shake, a locus with a point image is drawn on the image one-dimensionally. Spreads, that is, has a direction with a spread on a photographic image.

  In the field of digital photographic images, various methods have been conventionally proposed for restoring blurred images. If you know information such as blur direction and blur width when capturing a photographic image, you can restore it by applying a restoration filter such as a Wiener filter or inverse filter to the photographic image. There is widely known a method of providing an apparatus (for example, an acceleration sensor) that can be acquired in an imaging apparatus, acquiring information such as a blur direction and a blur width together with imaging, and performing repair based on the acquired information (for example, , See Patent Document 1).

  On the other hand, with the rapid spread of mobile phones, the functions of mobile phones have improved, and among them, the improvement of the functions of digital cameras attached to mobile phones (hereinafter referred to as mobile cameras) has been attracting attention. In recent years, the number of pixels of a portable camera has increased to one million, and the portable camera is used in the same way as a normal digital camera. Of course, commemorative photos when traveling with friends, as well as scenes of picking up favorite talents and athletes with a portable camera, are becoming commonplace. In such a background, a photographic image obtained by taking a picture with a mobile camera is not limited to being viewed on a monitor of a mobile phone. For example, it can be printed in the same manner as a photographic image obtained with a normal digital camera. It is increasing.

On the other hand, since the main body (mobile phone) is not manufactured exclusively for imaging, the portable camera has a problem of poor holdability during imaging. In addition, since the portable camera does not have a flash, the shutter speed is slower than a normal digital camera. For these reasons, camera shake is more likely to occur when shooting a subject with a portable camera than with a normal camera. Extreme camera shake can be confirmed on the monitor of the mobile phone, but small camera shake cannot be confirmed on the monitor. It is highly necessary to correct camera shake for the obtained photographic image.
Japanese Patent Laid-Open No. 2002-112099

  However, downsizing of mobile phones is one of the focus of competition among mobile phone manufacturers, along with their performance and cost, and a camera attached to a mobile phone is provided with a device that acquires the direction and width of blur. Since it is not realistic, there is a problem that the method as proposed in Patent Document 1 cannot be applied to a portable camera.

  In view of the above circumstances, the present invention provides an image processing method and apparatus capable of correcting camera shake in a digital photographic image without requiring a special apparatus in the imaging apparatus, and a program therefor. It is intended.

  In JP-A-7-121703, a deterioration function is set for a blurred image (an image with a blur), the blurred image is restored with a restoration filter corresponding to the set degradation function, and the restored image is displayed. A method is disclosed in which the evaluation is performed, and the deterioration function is reset based on the result of the evaluation, and the restoration is performed by repeating the repair, the evaluation, and the resetting of the deterioration function until a desired image quality is obtained. This method is a method of searching for a degradation function corresponding to a blurred image, and the process is different from the present invention.

According to a first image processing method of the present invention, a directional correction process for correcting camera shake is performed for a plurality of different directions on a digital photographic image having camera shake, and a plurality of images respectively corresponding to the directions. Correction processing for obtaining processed images of
An evaluation value calculation process for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed images;
A direction selection process for selecting a direction corresponding to a processed image for which an evaluation value representing the smallest degree of camera shake among the calculated evaluation values is calculated;
In the correction process, the direction correction process is performed on a plurality of different directions that include more components in the selected direction than any direction other than the direction selected by the direction selection process among the plurality of different directions. Repeat the direction setting process for setting as a plurality of new different directions until a predetermined condition is satisfied,
The method is characterized in that a processed image corresponding to the direction last selected by the direction selection process is acquired as a digital photo image in which the camera shake is corrected.

In the second image processing method of the present invention, a direction correction process for correcting camera shake is applied to a predetermined partial image in a digital photographic image having camera shake for each of a plurality of different directions. Correction processing for obtaining a plurality of processed partial images respectively corresponding to
An evaluation value calculation process for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed partial images;
A direction selection process for selecting a direction corresponding to the processed partial image for which the evaluation value indicating the degree of the smallest camera shake among the calculated evaluation values is calculated,
In the correction process, the direction correction process is performed on a plurality of different directions that include more components in the selected direction than any direction other than the direction selected by the direction selection process among the plurality of different directions. Repeat the direction setting process for setting as a plurality of new different directions until a predetermined condition is satisfied,
A directionality correction process is performed on the digital photographic image in a direction last selected by the direction selection process to obtain a digital photographic image in which the camera shake is corrected.

  Here, “a plurality of different directions” means directions of directionality correction processing performed for each correction processing on a digital photographic image (a partial image of the digital photographic image in the second image processing method). However, as the number of directions increases, the accuracy of camera shake correction increases.However, the processing time increases accordingly, so the number of directions depends on the priority ratio of camera shake correction accuracy and processing time. It is better to decide.

  Here, “directional correction” means correction that acts only in a predetermined direction. As the “direction correction processing”, a conventionally known correction method for improving the sharpness of an image, for example, USM (Unsharpness Masking) can be used, and a predetermined direction is used as a correction parameter. It is only necessary to set a directivity correction mask that acts on.

  As the “evaluation value”, for example, a score calculated based on the size of the edge width of each of a plurality of different directions detected in a digital photographic image can be considered. The edge width tends to concentrate on a relatively small value in a normal image without camera shake, while it tends to be dispersed to a relatively large value in an image with a large degree of camera shake. Therefore, for example, a higher score is calculated as the edge width is smaller, and a lower score is calculated as the edge width is smaller, so that the score is calculated according to the size of the edge width. A score can be calculated and the average score can be used as an evaluation value.

  The “predetermined condition” is any one of the number of repetitions of the processing described above reaching a predetermined number, the time required for the repetition reaching a predetermined time, and the evaluation value reaching a predetermined value. Can do.

  Which of the above conditions is adopted can be determined according to a priority matter in carrying out the image processing method of the present invention. For example, if priority is given to shortening the processing time, “required” `` Condition based on time '', `` Condition based on evaluation value '' if you want to prioritize correction accuracy, `` Condition based on the number of iterations '' if you want to prioritize both processing time reduction and correction accuracy It may be used. Of course, instead of using only one of the above conditions, a threshold may be set for each condition, and the process may be terminated when any one of the conditions is satisfied.

  When "different directions" are biased in a certain direction, direction correction processing is performed in a direction that is significantly different from the direction of actual camera shake in the digital photo image, and a sufficiently corrected digital photo image is obtained. Therefore, it is preferable that they are arranged at predetermined angles.

  The “plurality of different directions” may be arbitrary directions, but in order to simplify the algorithm, a predetermined plurality of predetermined directions, for example, eight directions as shown in FIG. 2 are selected. It may be set.

  In the second image processing method of the present invention, the “predetermined partial image” may be a partial image of a specific area determined in advance in the digital photographic image, or an area designating process is further provided. It may be a partial image of the area designated by. In the case of a general digital photographic image, the main part of the subject tends to exist at the center of the image, so that the specific area can be, for example, the central part of the digital photographic image.

A first image processing apparatus according to the present invention performs a directionality correction process for correcting the above-described camera shake for each of a plurality of different directions on a digital photographic image having a camera shake, and a plurality of images corresponding to the respective directions. Correction means for obtaining a processed image of
Evaluation value calculation means for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed images;
Direction selecting means for selecting a direction corresponding to the processed image from which the evaluation value representing the smallest degree of camera shake among the calculated evaluation values is calculated from the plurality of different directions;
A direction correction process is performed in the correction unit for a plurality of different directions including more components in the selected direction than any direction other than the direction selected by the direction selection unit among the plurality of different directions. Direction setting means for setting as a plurality of new different directions;
The direction correction processing by the correction unit, the evaluation value calculation by the evaluation value calculation unit, the direction selection by the direction selection unit, and the direction setting by the direction setting unit are repeated until a predetermined condition is satisfied. Iteration means;
Image acquisition means for acquiring a processed image corresponding to the direction last selected by the direction selection means as a digital photographic image in which the camera shake is corrected is provided.

The second image processing apparatus of the present invention performs direction correction processing for correcting the above-described camera shake for each of a plurality of different directions on a predetermined partial image in a digital photographic image having camera shake, and Correction means for obtaining a plurality of processed partial images respectively corresponding to
Evaluation value calculating means for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed partial images;
Direction selecting means for selecting a direction corresponding to the processed partial image for which the evaluation value representing the degree of the smallest camera shake among the calculated evaluation values is calculated from the plurality of different directions;
A direction correction process is performed in the correction unit for a plurality of different directions including more components in the selected direction than any direction other than the direction selected by the direction selection unit among the plurality of different directions. Direction setting means for setting as a plurality of new different directions;
The direction correction processing by the correction unit, the evaluation value calculation by the evaluation value calculation unit, the direction selection by the direction selection unit, and the direction setting by the direction setting unit are repeated until a predetermined condition is satisfied. Iteration means;
A second correction unit that performs a directionality correction process on the digital photographic image in a direction last selected by the direction selection unit, and acquires the digital photographic image in which the camera shake is corrected; It is characterized by.

  The “predetermined condition” is any one of the number of repetitions of the processing described above reaching a predetermined number, the time required for the repetition reaching a predetermined time, and the evaluation value reaching a predetermined value. Can do.

  When "different directions" are biased in a certain direction, direction correction processing is performed in a direction that is significantly different from the direction of actual camera shake in the digital photo image, and a sufficiently corrected digital photo image is obtained. Therefore, it is preferable that they are arranged at predetermined angles.

The first program of the present invention is a computer,
Correction means for performing a directional correction process for correcting the above-mentioned camera shake for each of a plurality of different directions with respect to a digital photographic image having camera shake, and obtaining a plurality of processed images respectively corresponding to the respective directions;
Evaluation value calculation means for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed images;
Direction selecting means for selecting a direction corresponding to the processed image from which the evaluation value representing the smallest degree of camera shake among the calculated evaluation values is calculated from the plurality of different directions;
A direction correction process is performed in the correction unit for a plurality of different directions including more components in the selected direction than any direction other than the direction selected by the direction selection unit among the plurality of different directions. Direction setting means for setting as a plurality of new different directions;
The direction correction processing by the correction unit, the evaluation value calculation by the evaluation value calculation unit, the direction selection by the direction selection unit, and the direction setting by the direction setting unit are repeated until a predetermined condition is satisfied. Iteration means;
It is a program for causing a processed image corresponding to the direction last selected by the direction selection unit to function as an image acquisition unit that acquires a digital photographic image in which the camera shake is corrected.

The second program of the present invention is a computer,
A direction correction process for correcting camera shake is performed on a predetermined partial image in a digital photographic image having camera shake for each of a plurality of different directions, and a plurality of processed partial images respectively corresponding to the respective directions are obtained. Correction means to obtain;
Evaluation value calculating means for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed partial images;
Direction selecting means for selecting a direction corresponding to the processed partial image for which the evaluation value representing the degree of the smallest camera shake among the calculated evaluation values is calculated from the plurality of different directions;
A direction correction process is performed in the correction unit for a plurality of different directions including more components in the selected direction than any direction other than the direction selected by the direction selection unit among the plurality of different directions. Direction setting means for setting as a plurality of new different directions;
The direction correction processing by the correction unit, the evaluation value calculation by the evaluation value calculation unit, the direction selection by the direction selection unit, and the direction setting by the direction setting unit are repeated until a predetermined condition is satisfied. Iteration means;
A direction correction process is performed on the digital photographic image in the direction last selected by the direction selection unit, and the digital photographic image functions as a second correction unit that acquires the digital photographic image in which the camera shake is corrected. It is a program.

  According to the first image processing method and apparatus and the program therefor according to the present invention, a directionality correction process is actually performed in a plurality of different directions on a digital photographic image having camera shake to evaluate the degree of camera shake. The camera shake is corrected by repeating the resetting of a new direction for which the direction correction process is performed by narrowing down in the direction in which the degree of the camera shake is reduced, so that the camera shake is corrected. Therefore, it is not necessary to provide a device for acquiring information regarding blurring during imaging. Therefore, camera shake correction can be performed without increasing the size of the image pickup apparatus, and there is a great merit particularly in the case of a digital camera attached to a mobile phone that requires downsizing.

  According to the second image processing method and apparatus of the present invention and the program therefor, a direction correction process is performed on a partial image instead of the entire digital photographic image having camera shake to evaluate the degree of camera shake, After searching for the direction of camera shake by repeating the setting of a new direction for which direction correction processing is performed by narrowing down the direction in which the degree of camera shake becomes smaller, the directionality in that direction with respect to the entire digital photo image Since correction processing is performed, while searching for the direction of camera shake, the amount of processing can be reduced by using a partial image with a small amount of data, and the time required for correcting camera shake can be shortened. Can be planned.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration according to an embodiment of a first image processing apparatus of the present invention. As shown in the figure, the image processing apparatus according to the present embodiment performs direction correction processing for correcting camera shake on a digital photographic image D having camera shake at predetermined angles as shown in FIG. A plurality of processed images corresponding to each of a plurality of predetermined different directions Ri (any one of i = 1, 2, 3,... 8) in the eight directions. The correction means 10 for obtaining Di, the evaluation value calculation means 20 for calculating the evaluation value Hi representing the degree of camera shake in each of the plurality of processed images Di, and the evaluation value Hi calculated from a plurality of different directions Ri. The direction selection means 30 for selecting the direction Rj (any one of the values that j = i can take) corresponding to the processed image for which the evaluation value representing the smallest degree of camera shake is calculated. Direction selection means 3 out of the directions Ri A plurality of different directions including more components of the selected direction than any direction other than the direction Rj selected by the above are defined as new different directions Ri for which the correction means 10 performs the directionality correction processing. Direction setting means 40 for setting from among the eight directions shown in FIG. 2, directionality correction processing by the correction means 10, evaluation value calculation by the evaluation value calculation means 20, direction selection by the direction selection means 30, and direction setting Repeating means 50 that repeats the setting of the direction by means 40 until a predetermined condition is satisfied, and the processed image corresponding to the direction last selected by the direction selecting means 30 is converted into a digital photographic image D with the above-mentioned camera shake corrected. And an image acquisition means 60 to acquire as'.

  In the present embodiment, the “predetermined condition” for ending the repetition process is when the number of repetition processes, more specifically, the number of direction selections by the direction selection unit 30 has reached three. .

  Next, the operation of the image processing apparatus configured as described above will be described.

  FIG. 3 is a diagram illustrating a processing flow of the image processing apparatus.

  The correction means 10 performs a directionality correction process on the digital photographic image D for each predetermined direction Ri, and obtains a processed image Di corresponding to each direction (step S11). In the first process, a directionality correction process is performed for each of the directions R1 and R5 to obtain processed images D1 and D5.

  In the direction correction process, USM (Unsharpness Masking), which is a correction method for improving the sharpness of an image, is used, and a predetermined direction (here, directions R1 and R5) is used as a correction parameter. ) To set the directivity correction mask.

  When the processed image Di is obtained by the correcting unit 10, the evaluation value calculating unit 20 calculates an evaluation value Hi representing the degree of camera shake in each processed image Di (step S12). In the first process, evaluation values H1 and H5 representing the degree of camera shake in the processed images D1 and D5 are calculated. As the evaluation value, an index value calculated based on the size of the edge width detected in the digital photographic image is used.

  The edge width tends to concentrate on a relatively small value in a normal image without camera shake, while it tends to be dispersed to a relatively large value in an image with a large degree of camera shake. Therefore, there is a correlation between the size of the edge width and the degree of camera shake, and an index value calculated based on the size of the edge width can be used as the evaluation value.

  Specific processing for calculating the evaluation value by the evaluation value calculating means 20 is as follows.

  First, an edge having a predetermined intensity or more is detected in each of the eight directions shown in FIG. 8 for the processed images (here, D1 and D5). A score corresponding to the size of the edge width is assigned to each detected edge. For example, threshold values W1 and W2 are set as W1 <W2, and the edge width is 100 points when W1 or less, 0 points when W2 or more, and between W1 and W2, the smaller the edge width between 100 and 0 points, the higher the edge width is. Scoring to be a score. Then, the average value of the scores of all detected edges is set as the evaluation value. In this way, it can be determined that the higher the evaluation value, the smaller the degree of camera shake.

  The threshold values W1 and W2 are based on, for example, a typical model of edge width histograms in a normal image and a camera shake image from a plurality of samples of a normal image without camera shake and a camera shake image with camera shake. To do so.

  When the evaluation value Hi is calculated by the evaluation value calculation means 20, the direction selection means 30 compares these evaluation values Hi with each other, detects the evaluation value with the smallest degree of camera shake, and the evaluation value is calculated. The direction Rj corresponding to the processed image is selected as the direction closer to the direction of camera shake (step S13). Here, it is assumed that the evaluation values H1 and H5 are calculated and the direction R5 is selected.

  When the direction Rj is selected by the direction selection unit 30, the repetition unit 50 determines whether or not the selection is a predetermined number of times, that is, the third selection. If it is less than three times, the repetition process is continued (step S14). Here, since it is still the first time, the repeated processing is continued.

  The direction setting unit 40 corrects a plurality of different directions including more components in the selected direction than any direction other than the direction Rj selected by the direction selection unit 30 among the plurality of different directions Ri. 2 are set from among the eight directions shown in FIG. 2 as a plurality of new different directions Ri to be subjected to the directionality correction processing (step S15). In the first process, three directions including the direction selected by the direction selecting means 30 and two directions skipped on both sides are set as new directions. Here, since the direction R5 is selected, three directions including the directions R3 and R7 in addition to the direction R5 are set.

  When a plurality of different directions Ri are newly set by the direction setting means 40, the correction means 10 performs the second process (step S11). Here, directionality correction processing is performed for each of the directions R3, R5, and R7 to obtain processed images D3, D5, and D7. Since the processed image D5 corresponding to the direction R5 has already been obtained in the first process, the directionality correction process in the direction R5 is omitted.

  When the processed image is obtained by the correcting unit 10, the evaluation value calculating unit 20 performs the second process (step S12). Here, since processed images D3, D5, and D7 are obtained, evaluation values H3, H5, and H7 are calculated. Since the evaluation value H5 has been calculated in the first process, the calculation of the evaluation value for the processed image D5 is omitted.

  When the evaluation value is calculated by the evaluation value calculation means 20, the direction selection means 30 performs the second process (step S13). Here, it is assumed that the evaluation values H3, H5, and H7 are calculated and the direction R3 is selected.

  Since the number of times of selection at this time is the second time and has not reached the third time, the repetitive means 50 continues the repetitive processing (step S14).

  Then, the direction setting means 40 performs the second process (step S15). If the selected direction is different between the first time and the second time, the direction setting means 40 is selected for the first time in addition to the second selected direction. Two directions including the direction between the selected direction and the direction selected for the second time are set. If the direction selected for the first time and the second time are the same, in addition to the selected direction, Three directions including the directions on both sides of the direction are set. Here, since the directions selected for the first time and the second time are different, two directions including the direction R4 are set in addition to the direction R3.

  When a new direction is set by the direction setting unit 40, the correction unit 10 performs a third process (step S11). Here, since the directions R3 and R4 are set as the new directions, processed images D3 and D4 are obtained. Since the processed image D3 corresponding to the direction R3 has already been obtained in the second process, the directionality correction process in the direction R3 is omitted.

  When the processed image is obtained by the correcting unit 10, the evaluation value calculating unit 20 performs the third process (step S12). Here, since processed images D3 and D4 are obtained, evaluation values H3 and H4 are calculated. Since the evaluation value H3 has been calculated in the second process, the calculation of the evaluation value for the processed image D3 is omitted.

  When the evaluation value is calculated by the evaluation value calculation unit 20, the direction selection unit 30 performs a third process (step S13). Here, it is assumed that the evaluation values H3 and H4 are calculated and the direction R4 is selected.

  Here, since the process by the direction selection unit 30 has reached the third time, the repetition unit 50 ends the repetition process in response thereto (step S14), and the image acquisition unit 60 receives it, and the direction selection unit 30 receives it. Thus, the processed image D4 corresponding to the direction R4 selected last is acquired as the digital photographic image D ′ in which the camera shake is corrected (step S16).

  As described above, according to the image processing apparatus according to the present embodiment, the directionality correction processing is actually performed in a plurality of different directions on the digital photographic image having camera shake to evaluate the degree of camera shake, and the degree of camera shake. In order to correct camera shake, we are trying to obtain a digital photo image with camera shake corrected by repeating the resetting of a new direction for which the direction correction processing is performed by narrowing down to a direction where the image becomes smaller. There is no need to provide a device for acquiring information regarding blurring during imaging. Therefore, camera shake correction can be performed without increasing the size of the image pickup apparatus, and there is a great merit particularly in the case of a digital camera attached to a mobile phone that requires downsizing.

  FIG. 4 is a block diagram showing a configuration according to an embodiment of the second image processing apparatus of the present invention. As shown in FIG. 2, the image processing apparatus according to the present embodiment performs a direction correction process for correcting the camera shake on a predetermined partial image d of a digital photographic image D having a camera shake as shown in FIG. , Applied to each of a plurality of different predetermined directions Ri (any one of i = 1, 2, 3,... 8) out of eight directions arranged at predetermined angles. Correction means 10 for obtaining a plurality of corresponding processed partial images di, evaluation value calculation means 20 for calculating an evaluation value hi representing the degree of camera shake in each of the plurality of processed partial images di, and a plurality of different directions Ri From among the calculated evaluation values hi, the direction Rj corresponding to the processed partial image for which the evaluation value representing the smallest degree of camera shake is calculated (any one of the values that j = i can take) is selected. Direction selecting means 30 to The correction means 10 performs a new direction correction process on a plurality of different directions including more components in the selected direction than any direction Rj selected by the direction selection means 30 among the directions Ri. As a plurality of different directions Ri, direction setting means 40 for setting from among the eight directions shown in FIG. 2, directionality correction processing by the correction means 10, calculation of evaluation values by the evaluation value calculation means 20, and direction selection means The direction selection unit 30 repeats the direction selection by the direction setting unit 40 and the direction setting by the direction setting unit 40 until a predetermined condition is satisfied, and the entire digital photograph image D is selected by the direction selection unit 30. And a second correction unit 70 that performs a direction correction process in the direction to acquire the digital photographic image D ′ in which the camera shake is corrected.

  In the present embodiment, the “predetermined partial image” to which the directionality correction process is performed is an image in a specific area at the center of the digital photographic image D.

  In addition, the operations of the correction unit 10, the evaluation value calculation unit 20, the direction selection unit 30, the direction setting unit 40, and the repetition unit 50 in the present embodiment are not the digital photographic image D to be processed but the digital photographic image D. Except for the fact that it is a partial image d, it is the same as the previous embodiment of the first image processing apparatus.

  Next, the operation of the image processing apparatus will be described.

  FIG. 5 is a diagram illustrating a processing flow of the image processing apparatus.

  The correction means 10 performs a directionality correction process on the partial image d of the digital photographic image D for each predetermined direction Ri to obtain a processed partial image di corresponding to each direction (step S21). In the first process, a directionality correction process is performed for each of the directions R1 and R5 shown in FIG. 2 to obtain processed partial images d1 and d5.

  As in the previous embodiment, USM is used for the directionality correction processing, and a directionality correction mask that acts in a predetermined direction (here, directions of directions 1 and 5) is set as a correction parameter.

  When the processed image di is obtained by the correcting unit 10, the evaluation value calculating unit 20 calculates an evaluation value hi representing the degree of camera shake in each processed partial image di (step S22). In the first process, evaluation values h1 and h5 representing the degree of camera shake in the processed partial images d1 and d5 are calculated. As the evaluation value, an index value calculated based on the size of the edge width of the edge detected in the digital photographic image is used as in the previous embodiment.

  When the evaluation value hi is calculated by the evaluation value calculation means 20, the direction selection means 30 compares these evaluation values hi with each other, detects the evaluation value with the smallest degree of camera shake, and the evaluation value is calculated. The direction Rj corresponding to the processed partial image is selected as a direction closer to the direction of camera shake (step S23). Here, it is assumed that the evaluation values h1 and h5 are calculated and the direction R5 is selected.

  When the direction Rj is selected by the direction selection unit 30, the repetition unit 50 determines whether or not the selection is a predetermined number of times, that is, the third selection. If it is less than three times, the repetition process is continued (step S24). Here, since it is still the first time, the repeated processing is continued.

  The direction setting unit 40 corrects a plurality of different directions including more components in the selected direction than any direction other than the direction Rj selected by the direction selection unit 30 among the plurality of different directions Ri. 2 are set from among the eight directions shown in FIG. 2 as a plurality of new different directions Ri to be subjected to the directionality correction processing (step S25). In the first process, three directions including the direction selected by the direction selecting means 30 and two directions skipped on both sides are set as new directions. Here, since the direction R5 is selected, three directions including the directions R3 and R7 in addition to the direction R5 are set.

  When a plurality of different directions Ri are newly set by the direction setting means 40, the correction means 10 performs the second process (step S21). Here, directionality correction processing is performed for each of the directions R3, R5, and R7 to obtain processed partial images d3, d5, and d7. Since the processed partial image d5 corresponding to the direction R5 has already been obtained in the first process, the directionality correction process in the direction R5 is omitted.

  When the processed partial image is obtained by the correcting unit 10, the evaluation value calculating unit 20 performs the second process (step S22). Here, since the processed partial images d3, d5, d7 are obtained, the evaluation values h3, h5, h7 are calculated. Since the evaluation value h5 has been calculated in the first process, the calculation of the evaluation value for the processed partial image d5 is omitted.

  When the evaluation value is calculated by the evaluation value calculation means 20, the direction selection means 30 performs the second process (step S23). Here, it is assumed that the evaluation values h3, h5, and h7 are calculated and the direction R3 is selected.

  Since the number of selections at this time is the second time and has not reached the third time, the repetitive means 50 continues the repetitive processing (step S24).

  Then, the direction setting means 40 performs the second process (step S25). If the selected direction is different between the first time and the second time, the direction setting means 40 is selected for the first time in addition to the direction selected for the second time. Two directions including the direction between the selected direction and the direction selected for the second time are set. If the direction selected for the first time and the second time are the same, in addition to the selected direction, Three directions are set by adding the directions on both sides of the direction. Here, since the directions selected for the first time and the second time are different, two directions including the direction R4 are set in addition to the direction R3.

  When a new direction is set by the direction setting unit 40, the correction unit 10 performs a third process (step S21). Here, since the directions R3 and R4 are set as the new directions, processed partial images d3 and d4 are obtained. Since the processed partial image d3 corresponding to the direction R3 has already been obtained in the second process, the directionality correction process in the direction R3 is omitted.

  When the processed partial image is obtained by the correcting unit 10, the evaluation value calculating unit 20 performs the third process (step S22). Since the processed partial images d3 and d4 are obtained here, the evaluation values h3 and h4 are calculated. Since the evaluation value h3 has been calculated in the second process, the calculation of the evaluation value for the processed partial image d3 is omitted.

  When the evaluation value is calculated by the evaluation value calculation means 20, the direction selection means 30 performs the third process (step S23). Here, the evaluation values h3 and h4 are calculated, and the direction R4 is selected.

  Here, since the process by the direction selection means 30 has reached the third time, the repetition means 50 ends the repetition process in response thereto (step S14), and the second correction means 70 receives it and receives the digital photograph. The entire image D is subjected to a directionality correction process in the direction R4 last selected by the direction selection unit 30 to obtain a digital photo image D ′ in which camera shake is corrected (step S26).

  As described above, according to the image processing apparatus according to the present embodiment, the direction correction process is performed on the partial image instead of the entire digital photographic image having the camera shake, the degree of the camera shake is evaluated, and the degree of the camera shake is determined. After searching for the direction of camera shake by repeatedly setting a new direction to which the direction correction processing is performed by narrowing down to a smaller direction, the direction correction processing is performed on the entire digital photo image in that direction. Therefore, while searching for the direction of camera shake, the amount of processing can be reduced by using a partial image with a small amount of data, and the time required for correcting camera shake can be shortened. .

  In the above two embodiments, the direction in which the direction correction process is performed is eight directions arranged at predetermined angles as shown in FIG. 2, so that the actual camera shake direction and the direction correction process are performed. There is no significant deviation from the direction. Further, the present invention is not limited to these eight directions, and if accuracy is required, the directions may be further subdivided into 16 directions (directions set at an angle of 11.25 °).

  In the two embodiments, the direction in which the direction correction process is performed by the correction unit 10 is the directions R1 and R5 in the first process, and the direction selected by the direction selection unit 30 in the second process. And three directions including two directions skipped on both sides are set as new directions. Of course, the direction is not limited to this method. For example, in the first process, the direction is R2, R5, and R8 may be set, and then the three directions including the direction selected by the direction selection unit 30 and the directions on both sides thereof may be set as new directions.

  In the above embodiment, a symmetric filter such as a one-dimensional USM is used. However, an asymmetric filter may be used. In this case, a direction opposite to the eight directions shown in FIG. The same processing may be performed in such 16 directions (directions set at intervals of 22.5 °).

  In the above two embodiments, the evaluation value is calculated by the evaluation value calculation means 20 after the directionality correction processing for each direction by the correction means 10. For each direction, the directionality correction process and the evaluation value may be calculated simultaneously.

The block diagram which shows the structure by one Embodiment of the 1st image processing apparatus of this invention. The figure which shows the direction used when performing a directionality correction process The figure which shows the processing flow in embodiment of the 1st image processing apparatus. The block diagram which shows the structure by one Embodiment of the 2nd image processing apparatus of this invention. The figure which shows the processing flow in embodiment of the 2nd image processing apparatus. The figure which shows the direction used when performing the directionality correction process by an asymmetric filter

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Correction | amendment means 20 Evaluation value calculation means 30 Direction selection means 40 Direction setting means 50 Repetition means 60 Image acquisition means 70 2nd correction means D Digital photograph image Di Processed image D 'Shake-corrected digital photograph image d Digital photograph image Partial image di processed partial image Hi evaluation value representing the degree of camera shake in the processed image hi evaluation value representing the degree of camera shake in the processed partial image Ri direction to perform the directionality correction process Rj direction selected by the direction selection means

Claims (10)

Correction processing for obtaining a plurality of processed images respectively corresponding to the respective directions by performing direction correction processing for correcting the camera shake for each of a plurality of different directions with respect to a digital photographic image having camera shake;
An evaluation value calculation process for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed images;
A direction selection process for selecting a direction corresponding to the processed image from which the evaluation value representing the degree of the smallest camera shake among the calculated evaluation values is calculated, from among the plurality of different directions;
Among the plurality of different directions, a plurality of different directions containing more components of the selected direction than any direction other than the direction selected by the direction selection processing are subjected to the directionality correction processing in the correction processing. It repeats the direction setting processing to set as a plurality of new different directions to be applied until a predetermined condition is satisfied,
An image processing method, wherein the processed image corresponding to a direction last selected by the direction selection process is acquired as the digital photographic image in which the camera shake is corrected. A direction correction process for correcting the camera shake is performed on a predetermined partial image in a digital photographic image having camera shake in a plurality of different directions, and a plurality of processed partial images respectively corresponding to the directions are obtained. Correction processing to obtain,
An evaluation value calculation process for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed partial images;
A direction selection process for selecting a direction corresponding to the processed partial image for which the evaluation value representing the degree of the smallest camera shake among the calculated evaluation values is calculated from the plurality of different directions;
Among the plurality of different directions, a plurality of different directions containing more components of the selected direction than any direction other than the direction selected by the direction selection processing are subjected to the directionality correction processing in the correction processing. It repeats the direction setting processing to set as a plurality of new different directions to be applied until a predetermined condition is satisfied,
An image processing method, wherein the direction correction process is performed on the digital photographic image in a direction last selected by the direction selection process to obtain the digital photographic image in which the camera shake is corrected. .   The predetermined condition is that the number of repetitions reaches a predetermined number, the time required for the repetition reaches a predetermined time, or the evaluation value reaches a predetermined value. Item 3. The image processing method according to Item 1 or 2.   The image processing method according to claim 1, wherein the plurality of different directions are arranged at predetermined angles. Correction means for performing a directional correction process for correcting the camera shake for each of a plurality of different directions with respect to a digital photographic image having camera shake, and obtaining a plurality of processed images respectively corresponding to the directions;
Evaluation value calculating means for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed images;
Direction selecting means for selecting a direction corresponding to the processed image from which the evaluation value representing the degree of the smallest camera shake among the calculated evaluation values is calculated from the plurality of different directions;
Among the plurality of different directions, a plurality of different directions including more components of the selected direction than any direction other than the direction selected by the direction selection unit are subjected to the directionality correction processing in the correction unit. Direction setting means for setting as a plurality of new different directions to be applied;
The direction correction processing by the correction means, the evaluation value calculation by the evaluation value calculation means, the direction selection by the direction selection means, and the direction setting by the direction setting means are repeated until a predetermined condition is satisfied. Iteration means;
An image processing apparatus comprising: an image acquisition unit that acquires the processed image corresponding to the direction last selected by the direction selection unit as the digital photographic image in which the camera shake is corrected. A direction correction process for correcting the camera shake is performed on a predetermined partial image in a digital photographic image having camera shake in a plurality of different directions, and a plurality of processed partial images respectively corresponding to the directions are obtained. Correction means to obtain;
Evaluation value calculating means for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed partial images;
Direction selection means for selecting a direction corresponding to the processed partial image for which the evaluation value representing the degree of the smallest camera shake among the calculated evaluation values is calculated from the plurality of different directions;
Among the plurality of different directions, a plurality of different directions including more components of the selected direction than any direction other than the direction selected by the direction selection unit are subjected to the directionality correction processing in the correction unit. Direction setting means for setting as a plurality of new different directions to be applied;
The direction correction processing by the correction means, the evaluation value calculation by the evaluation value calculation means, the direction selection by the direction selection means, and the direction setting by the direction setting means are repeated until a predetermined condition is satisfied. Iteration means;
A second correction unit that performs the direction correction process on the digital photographic image in a direction last selected by the direction selection unit, and acquires the digital photographic image in which the camera shake is corrected. An image processing apparatus characterized by that.   The predetermined condition is that the number of repetitions reaches a predetermined number, the time required for the repetition reaches a predetermined time, or the evaluation value reaches a predetermined value. Item 7. The image processing apparatus according to Item 5 or 6.   The image processing apparatus according to claim 5, wherein the plurality of different directions are arranged at predetermined angles. Computer
Correction means for performing a directional correction process for correcting the camera shake for each of a plurality of different directions with respect to a digital photographic image having camera shake, and obtaining a plurality of processed images respectively corresponding to the directions;
Evaluation value calculating means for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed images;
Direction selecting means for selecting a direction corresponding to the processed image from which the evaluation value representing the degree of the smallest camera shake among the calculated evaluation values is calculated from the plurality of different directions;
Among the plurality of different directions, a plurality of different directions including more components of the selected direction than any direction other than the direction selected by the direction selection unit are subjected to the directionality correction processing in the correction unit. Direction setting means for setting as a plurality of new different directions to be applied;
The direction correction processing by the correction means, the evaluation value calculation by the evaluation value calculation means, the direction selection by the direction selection means, and the direction setting by the direction setting means are repeated until a predetermined condition is satisfied. Iteration means;
A program for causing the processed image corresponding to the direction last selected by the direction selection unit to function as an image acquisition unit that acquires the digital photographic image with the camera shake corrected. Computer
A direction correction process for correcting the camera shake is performed on a predetermined partial image in a digital photographic image having camera shake in a plurality of different directions, and a plurality of processed partial images respectively corresponding to the directions are obtained. Correction means to obtain;
Evaluation value calculation means for calculating an evaluation value representing the degree of camera shake in each of the plurality of processed partial images;
Direction selection means for selecting a direction corresponding to the processed partial image for which the evaluation value representing the degree of the smallest camera shake among the calculated evaluation values is calculated from the plurality of different directions;
Among the plurality of different directions, a plurality of different directions including more components of the selected direction than any direction other than the direction selected by the direction selection unit are subjected to the directionality correction processing in the correction unit. Direction setting means for setting as a plurality of new different directions to be applied;
The direction correction processing by the correction means, the evaluation value calculation by the evaluation value calculation means, the direction selection by the direction selection means, and the direction setting by the direction setting means are repeated until a predetermined condition is satisfied. Iteration means;
The direction correction process is performed on the digital photographic image in the direction last selected by the direction selection unit to function as a second correction unit that acquires the digital photographic image in which the camera shake is corrected. Program for.

Images