CN101778222B - Image processing apparatus - Google Patents

Abstract

An image processing device includes an image acquisition section for acquiring an image picked up by an image pickup element and defect information creation section for creating defect information on an acquired image according to a value of a pixel in concern and an average value of values of a plurality of pixels in a predetermined range containing the pixel in concern.

Description

Image processing apparatus

The application be international filing date be December 25, international application no in 2003 be 200380107791.8 for PCT/JP03/16797, national applications number, denomination of invention divides an application for the application for a patent for invention of " image processing apparatus and image processing program ".

The disclosure of following basis for priority application is integrated into here as citation.

Japanese patent application 2002 No. 379719 (application on December 27th, 2002)

Japanese patent application 2002 No. 379720 (application on December 27th, 2002)

Japanese patent application 2002 No. 379721 (application on December 27th, 2002)

Japanese patent application 2003 No. 307355 (application on 08 29th, 2003)

Japanese patent application 2003 No. 307356 (application on 08 29th, 2003)

Japanese patent application 2003 No. 307357 (application on 08 29th, 2003)

Technical field

The present invention relates in to remove the image processing apparatus of the influence of dust etc. by shot image data such as electronic cameras.

Background technology

In the past, opened the spy and to disclose following technology in the flat 9-51459 communique: to be blended into the effect of dust in the optical system in order proofreading and correct when making video camera, to take the white figure of each f-number in advance, and record control information.In addition; Opening flat 10-294870 communique and spy the spy opens in the flat 11-27475 communique and discloses following technology: in the photocopier field; Countermeasure as the dust of the possibility with frequent variation is taken into the white reference data with uniform reflecting surface before reading original copy, carry out dust and detect.And then, at USP6,195; Following method is disclosed in No. 161: in the scanner field, in order to play the effect that replaces these white reference data, infrared light transducer[sensor is installed; Obtain transmisivity data simultaneously with visible data, the deamplification of the transmissivity that obtains causing by film defects.

Summary of the invention

But, in existing camera, be adsorbed on fixedly dust on the optics in the time of only will making, and do not have to consider the dust that changes along with the process of frequency of utilization, time as object.In the slr camera of the lens changeable formula that begins now to popularize, particularly owing to the anterior optics of imaging apparatus exposes, so writing of time dependent dust becomes very big problem easily.

On the other hand, before this scanning or obtain the dust data simultaneously, corresponding in photocopier or scanner with vicissitudinous dust on the time.But, different with the structure of camera, have uniform lighting apparatus with respect to original copy that is positioned at fixed range or film surface, and then, can be relatively easy to obtain transmisivity data through fully uniform reflecting surface being installed or new infrared illumination equipment being set.But, in electronic camera, except inspection, be difficult to obtain this transmisivity data of even face completely during manufacturing.

In addition, photocopier, scanner are essentially fixed optical system, need not to consider that the variation dust along with optical system changes.On the other hand, not corresponding in existing video camera with the optical condition variation beyond the f-number.

The present invention provide a kind of can be from by image processing apparatus and the image processing program of removing effect of dust etc. suitable the shot image data such as electronic camera.

According to the 1st mode of the present invention, a kind of image processing apparatus has: image acquiring unit obtains the image of being taken by imaging apparatus; With defect information preparing department, in the image that obtains, according to the mean value of a plurality of pixel values in the value of object pixel and the preset range that comprises this object pixel, the defect information in the construction drawing picture.

According to the 2nd mode of the present invention; Preferably; In the image processing apparatus of the 1st mode, defect information preparing department has the calculating part of comparing, and according to the making defect information of calculating that compares; The wherein said calculating part that compares, the comparing of mean value of calculating a plurality of pixel values in object pixel and the preset range that comprises this object pixel.

According to the 3rd mode of the present invention, preferably, at any one image processing apparatus of the 1st mode or the 2nd mode, defect image preparing department makes defect information to the zone of in image, satisfying predetermined condition.

According to the 4th mode of the present invention, preferably, in the image processing apparatus of any mode, also has correction portion, according to the defective in the defect information correcting image the 1st～3.

According to the 5th mode of the present invention, preferably, in the image processing apparatus of the 2nd mode, also have correction portion, according to the defective in the defect information correcting image, correction portion multiply by corresponding pixel value with the reciprocal value that compares and proofreaies and correct.

According to the 6th mode of the present invention, preferably, in the image processing apparatus of the 1st mode; Image acquiring unit obtains a plurality of images by the imaging apparatus shooting, defect information preparing department; A plurality of images that utilization obtains are made the defect information in the arbitrary image in a plurality of images.

According to the 7th mode of the present invention, preferably, in the image processing apparatus of the 1st mode; Image acquiring unit obtains a plurality of images by the imaging apparatus shooting, defect information preparing department; A plurality of images that utilization obtains are made the corresponding defect information of integral image with a plurality of images.

According to the 8th mode of the present invention, a kind of image processing apparatus has: image acquiring unit obtains the benchmark image of taking through optical system; With defect information preparing department, in the benchmark image that obtains,, make the defect information in the benchmark image according to the mean value of a plurality of pixel values in the value of object pixel and the preset range that comprises this object pixel.

According to the 9th mode of the present invention; Preferably; In the image processing apparatus of the 8th mode, defect information preparing department has the calculating part of comparing, and according to the making defect information of calculating that compares; The calculating part that wherein compares, the comparing of mean value of calculating a plurality of pixel values in object pixel and the preset range that comprises this object pixel.

According to the 10th mode of the present invention, preferably, in the image processing apparatus of any mode the 8th～9; Image acquiring unit obtains the calibration object image through the optical system shooting, also has correction portion; According to the defect information in the benchmark image, the defective in the corrected object images.

According to the 11st mode of the present invention; Preferably; In the image processing apparatus of the 10th mode, when taking benchmark image and calibration object image through the optical system of the identical in fact optical condition of f-number and pupil location, correction portion; Directly utilize the defect information that generates, proofread and correct the value of the pixel that constitutes the calibration object image.

According to the 12nd mode of the present invention, preferably, in the image processing apparatus of the 10th mode; Also has the defect information transformation component; With corresponding as at least one of the f-number of the optical condition of optical system and pupil location, the conversion defect information is when the optical system of at least one the different optical condition through f-number and pupil location is taken benchmark image and calibration object image; Utilize the defect information of conversion, proofread and correct the value of the pixel that constitutes the calibration object image.

According to the 13rd mode of the present invention, preferably, in the image processing apparatus of the 9th mode, also have correction portion, the value of pixel that the reciprocal value that compares of benchmark image multiply by the correspondence of calibration object image is proofreaied and correct.

According to the 14th mode of the present invention, preferably, in the image processing apparatus of any mode the 2nd, 9, the calculating part that compares when comparing of calculating is included in when clipping 1 preset range, is set at 1 with comparing of calculating.

According to the 15th mode of the present invention, preferably, in the image processing apparatus of the 14th mode, the calculating part that compares makes comparing of calculating is set at 1 preset range, sets up related with the standard deviation value that compares of calculating.

According to the 16th mode of the present invention, preferably, in the image processing apparatus of any mode, comprise the preset range of object pixel the 1st, 8, be the scope also bigger than the defect area that in image or benchmark image, generates.

According to the 17th mode of the present invention, preferably, in the image processing apparatus of the 10th mode, image acquiring unit obtains the benchmark image of taking in the scheduled time before and after the shooting of calibration object image.

According to the 18th mode of the present invention, preferably, in the image processing apparatus of the 17th mode, image acquiring unit obtains at the benchmark image of taking with the interior time with the second near time of the shooting time of calibration object image.

According to the 19th mode of the present invention, a kind of image processing apparatus has: image acquiring unit, obtain the image that imaging apparatus that use can beam split in a plurality of colors is taken; Luminance signal generation portion is from a plurality of chrominance signals generation luminance signals of image; With defect information preparing department, according to the luminance signal of the image that generates, the defect information in the construction drawing picture.

According to the 20th mode of the present invention, preferably, in the image processing apparatus of the 19th mode, also have correction portion, utilize the value of each colour component of the defect pixel in the defect information correcting image.

According to the 21st mode of the present invention; Preferably; In the image processing apparatus of the 19th mode, has defect information preparing department, in the image that obtains; According to the value of the luminance signal of the generation of object pixel with comprise the mean value of luminance signal of the generation of a plurality of pixels in the preset range of this object pixel, the defect information in the construction drawing picture.

According to the 22nd mode of the present invention; Preferably, in the image processing apparatus of the 21st mode, defect information preparing department; Has the calculating part of comparing; And according to the making defect information of calculating that compares, the wherein said calculating part that compares calculates the mean value of luminance signal of the generation of a plurality of pixels in value and the preset range that comprises this object pixel of luminance signal of generation of object pixel.

According to the 23rd mode of the present invention, preferably, in the image processing apparatus of the 22nd mode, also have correction portion, the value that the reciprocal value that compares multiply by each colour component of corresponding pixel is proofreaied and correct.

According to the 24th mode of the present invention, preferably, in the image processing apparatus of the 19th mode; Image acquiring unit obtains a plurality of images by the imaging apparatus shooting, luminance signal generation portion; A plurality of images for obtaining generate luminance signal; Defect information preparing department, the luminance signal of a plurality of images that utilization generates, the interior defect information of image arbitrarily of making a plurality of images.

According to the 25th mode of the present invention, preferably, in the image processing apparatus of the 19th mode; Image acquiring unit obtains a plurality of images by the imaging apparatus shooting, luminance signal generation portion; A plurality of images for obtaining generate luminance signal; Defect information preparing department utilizes a plurality of images that obtain, and makes the defect information corresponding with the integral image of a plurality of images.

According to the 26th mode of the present invention, preferably, in the image processing apparatus of the 1st mode, imaging apparatus is through the optical system photographic images, and defect information is the information of the projection image of the defective in the light path way that in image, generates.

According to the 27th mode of the present invention, preferably, in the image processing apparatus of the 26th mode, defect information preparing department, make simultaneously and record and light path way in the position information relevant of projection image of defective with intensity.

According to the 28th mode of the present invention; Preferably, in the image processing apparatus of the 27th mode, defect information preparing department; In the preset range of each object pixel being calculated mean value, move, make the relevant information of projection image continuously with the defective of light path on the way.

According to the 29th mode of the present invention, preferably, in the image processing apparatus of the 4th mode, correction portion utilizes the original signal value of calibration object location of pixels to try to achieve corrected value.

According to the 30th mode of the present invention, preferably, in the image processing apparatus of the 8th mode, defect information is the information of the projection image of the defective in the light path way that in image, generates.

According to the 31st mode of the present invention, preferably, in the image processing apparatus of the 30th mode, defect information preparing department, make simultaneously and record and light path way in the position information relevant of projection image of defective with intensity.

According to the 32nd mode of the present invention; Preferably, in the image processing apparatus of the 31st mode, defect information preparing department; In the preset range of each object pixel being calculated mean value, move, make the relevant information of intensity continuously with the projection image of the defective of light path on the way.

According to the 33rd mode of the present invention, preferably, in the image processing apparatus of the 10th mode, correction portion utilizes the original signal value of calibration object location of pixels to try to achieve corrected value.

According to the 34th mode of the present invention, preferably, in the image processing apparatus of the 15th mode, the calculating part that compares with making comparing of calculating be set at 1 preset range, is set at ± scope of (3 * standard deviation value).

According to the 35th mode of the present invention, preferably, in the image processing apparatus of the 26th mode, comprise the preset range of object pixel, be the big scope of projection image than the defective in the light path way that in image, generates.

According to the 36th mode of the present invention, preferably, in the image processing apparatus of the 30th mode, comprise the preset range of object pixel, be the big scope of projection image than the defective in the light path way that in benchmark image, generates.

According to the 37th mode of the present invention, a kind of image processing apparatus has: image acquiring unit obtains first image of taking through optical system and second image of taking with the optical condition different with first image; With defect information preparing department, utilize first image and second image, make the defect information that is comprised in first image or second image.

According to the 38th mode of the present invention, preferably, in the image processing apparatus of the 37th mode, have correction portion, utilize defect information, proofread and correct the defective in first image or second image.

According to the 39th mode of the present invention, preferably, in the image processing apparatus of the 37th mode, between first image and second image, at least one optical condition of f-number and pupil location is different.

According to the 40th mode of the present invention; Preferably; In the image processing apparatus of any mode the 37th～39, defect information preparing department has the optical condition transformation component, inconsistent for the optical condition of eliminating first image and second image; Carry out conversion, so that at least one image of first image and second image is consistent under certain optical condition.

According to the 41st mode of the present invention; Preferably; In the image processing apparatus of the 40th mode, the optical condition transformation component, at the f-number of optical condition not simultaneously; Picture element signal based on first image or second image is carried out low pass filter handle, be transformed to the state of the defective of supposition when identical f-number.

According to the 42nd mode of the present invention, preferably, in the image processing apparatus of the 41st mode, the optical condition transformation component, utilizing in fact all, the low pass filter of equal weight carries out conversion.

According to the 43rd mode of the present invention; Preferably, in the image processing apparatus of the 40th mode, the optical condition transformation component; In the pupil location of optical condition not simultaneously; To picture element signal, carry out being transformed to the state that supposition is positioned at the defective of identical pupil location from the optical axis center of optical system shifting processing to the skew of radius vector direction based on first image or second image.

According to the 44th mode of the present invention, preferably, in the image processing apparatus of the 43rd mode, the optical condition transformation component, carry out along with from optical axis center away from and shifting processing that the amount that squints in the radius vector direction increases.

According to the 45th mode of the present invention; Preferably; In the image processing apparatus of any mode the 43rd～44, the optical condition transformation component supposes that the foreign matter of the reason that becomes defective is positioned at the position that preset distance is arranged along the shooting mask of optical axis and optical system; Predict the computing of shift amount, carry out shifting processing.

According to the 46th mode of the present invention, preferably, in the image processing apparatus of the 37th mode, any one of first image and second image is that another is the benchmark image that is used to make defect information as the calibration object image of the object of proofreading and correct.

According to the 47th mode of the present invention; Preferably; In the image processing apparatus of the 37th mode, first image and second image all are the calibration object images as the object of proofreading and correct, defect information preparing department; Utilize first image and second image, process first image and the common defect information of second image.

According to the 48th mode of the present invention; Preferably; In the image processing apparatus of the 47th mode, defect information preparing department has the optical condition transformation component, inconsistent for the optical condition of eliminating first image and second image; Carry out conversion, so that at least one image of first image and second image is consistent under certain optical condition.

According to the 49th mode of the present invention, preferably, in the image processing apparatus of the 46th mode, image acquiring unit obtains the benchmark image of taking with the f-number that in the iris ring value of optical system, turns to minimum state.

According to the 50th mode of the present invention, preferably, in the image processing apparatus of the 37th mode; Defect information preparing department; In the image that obtains, according to the mean value of the value of a plurality of pixels in the value of object pixel and the preset range that comprises this object pixel, the defect information in the construction drawing picture.

According to the 51st mode of the present invention, preferably, in the image processing apparatus of any mode the 46th, 49, image acquiring unit obtains the benchmark image of taking in the scheduled time before and after the shooting of calibration object image.

According to the 52nd mode of the present invention, a kind of image processing apparatus has: image acquiring unit obtains first image of taking through optical system and second image of taking with the optical condition different with first image; With correction portion, utilize first image and second image, proofread and correct the defective that is comprised in first image or second image.

According to the 53rd mode of the present invention, preferably, in the image processing apparatus of the 52nd mode, between first image and second image, at least one optical condition of f-number and pupil location is different.

According to the 54th mode of the present invention, a kind of image processing apparatus has: image acquiring unit obtains the photographic images of being taken by imaging apparatus; Flat extraction portion extracts the flat zone out in the photographic images that obtains; With defect information preparing department, make the subregional defect information in par of extracting out.

According to the 55th mode of the present invention, preferably, in the image processing apparatus of the 54th mode, also have correction portion, according to defect information, proofread and correct the subregional image in par.

According to the 56th mode of the present invention; Preferably; In the image processing apparatus of any mode the 54th～55; In the subregional image in par,, make the subregional defect information in par according to the mean value of a plurality of pixels in the value of object pixel and the preset range that comprises this object pixel.

According to the 57th mode of the present invention; Preferably, in the image processing apparatus of the 56th mode, defect information preparing department; Has the calculating part of comparing; And make the subregional defect information in par according to comparing of calculating, the comparing of the mean value of a plurality of pixels in the calculating part that wherein compares, the value of calculating object pixel and the preset range that comprises this object pixel.

According to the 58th mode of the present invention; Preferably, in the image processing apparatus of the 55th mode, also has the calculating part of comparing; Calculate to constitute in the pixel of the subregional image in par, the value of object pixel and comprise the comparing of mean value of a plurality of pixels in the preset range of this object pixel; Defect information preparing department compares according to what calculate, makes the subregional defect information in par; Correction portion will the reciprocal value that compare corresponding with the pixel of the subregional image in par multiply by the subregional image corresponding pixel value in par and proofreaies and correct.

According to the 59th mode of the present invention; Preferably; In the image processing apparatus of the 58th mode; Correction portion handles carrying out low pass as comparing of the defect information of processing, and the reciprocal value that compares after low pass that will be corresponding with the pixel of the subregional image in par is handled multiply by the subregional image corresponding pixel value in par proofreaies and correct.

According to the 60th mode of the present invention, preferably, in the image processing apparatus of any mode the 54th～59, flat extraction portion carries out the edge and extracts out in photographic images, will not extract the zone at edge out and extract out as the flat zone.

According to the 61st mode of the present invention, preferably, in the image processing apparatus of the 54th mode, flat extraction portion has the greyscale transformation portion that photographic images is carried out greyscale transformation, and the photographic images after the greyscale transformation is carried out the subregional extraction in par.

According to the 62nd mode of the present invention, preferably, in the image processing apparatus of the 61st mode, greyscale transformation portion when the gray scale of photographic images is linear signal, is transformed to nonlinear properties.

According to the 63rd mode of the present invention, preferably, in the image processing apparatus of the 62nd mode, the conversion of gray scale of gray scale, the compression high brightness side of low-light level side is amplified by greyscale transformation portion.

According to the 64th mode of the present invention, preferably, in the image processing apparatus of any mode the 62nd～63, greyscale transformation portion carries out conversion according to power function.

According to the 65th mode of the present invention, preferably, in the image processing apparatus of the 64th mode, power function is a square root function.

According to the 66th mode of the present invention, preferably, in the image processing apparatus of any mode the 60th～65, the edge is extracted out, between object pixel and neighboring pixel, through a plurality of directions are carried out being extracted out by the calculus of differences that a plurality of distances constitute.

According to the 67th mode of the present invention; Preferably; In the image processing apparatus of any mode the 55th, 58,59, have the brightness degree judging part, whether the brightness degree of judging photographic images is in predetermined brightness or more than it; Correction portion, to brightness degree in predetermined brightness or more than it and be that proofread and correct in subregional zone, par.

According to the 68th mode of the present invention, preferably, in the image processing apparatus of the 54th mode, also have: benchmark image obtains portion, obtains the benchmark image of being taken by imaging apparatus; With benchmark image defect information preparing department, make the defect information of benchmark image, defect information preparing department, the area information of the defect information of combination use benchmark image and the area information of smooth subregion are made the subregional defect information in par.

According to the 69th mode of the present invention; Preferably; In the image processing apparatus of the 68th mode, flat extraction portion is not even be used as the zone that extract out in the flat zone; Under the situation of the defect area that the defect information that is benchmark image is represented, this defect area is extracted out as the flat zone.

According to the 70th mode of the present invention; Preferably, in the image processing apparatus of any mode the 68th～69, defect information preparing department; The defect area that the defect information that satisfies benchmark image is represented and the both sides' of smooth subregion zone makes defect information.

According to the 71st mode of the present invention; Preferably, in the image processing apparatus of the 68th mode, also has the defect information transformation component; When photographic images and benchmark image take be optical condition not simultaneously; The defect information of benchmark image is transformed to the defect information with the defect information equivalence of the benchmark image of under the optical condition identical with photographic images, taking, correction portion, utilize conversion the defect information of benchmark image.

According to the 72nd mode of the present invention; Preferably, in the image processing apparatus of any mode, also has the defect information transformation component the 69th～70; Optical condition when photographic images and benchmark image shooting is not simultaneously; The defect information of benchmark image is transformed to the defect information with the defect information equivalence of the benchmark image of under the optical condition identical with photographic images, taking, flat extraction portion and correction portion, utilize conversion the defect information of benchmark image.

According to the 73rd mode of the present invention; Preferably; In the image processing apparatus of any mode the 71st～72, correction portion, the defect information mapping fault of consideration defect information transformation component; With the defect area that the defect information of benchmark image is represented, amplify defect information mapping fault degree at least and utilize.

According to the 74th mode of the present invention, preferably, in the image processing apparatus of the 54th mode; Image acquiring unit obtains a plurality of photographic images by the imaging apparatus shooting, flat extraction portion; In a plurality of photographic images, extract the flat zone out, defect information preparing department; Use the subregional image in par of a plurality of images of extracting out, the subregional defect information in par of making the image arbitrarily of a plurality of images.

According to the 75th mode of the present invention, preferably, in the image processing apparatus of the 54th mode; Image acquiring unit obtains a plurality of photographic images by the imaging apparatus shooting, flat extraction portion; In a plurality of photographic images, extract the flat zone out, defect information preparing department; Use the subregional image in par of a plurality of images of extracting out, make the defect information corresponding with the integral image of a plurality of images.

According to the 76th mode of the present invention; Preferably; In the image processing apparatus of any mode the 74th, 75; In the subregional image in par,, make the subregional defect information in par according to the mean value of a plurality of pixels in the value of object pixel and the preset range that comprises this object pixel.

According to the 77th mode of the present invention; Preferably; In the image processing apparatus of the 76th mode, defect information preparing department has the calculating part of comparing, and makes the subregional defect information in par according to comparing of calculating; Comparing of the mean value of a plurality of pixels in the calculating part that wherein compares, the value of calculating object pixel and the preset range that comprises this object pixel.

According to the 78th mode of the present invention; Preferably; In the image processing apparatus of the 69th mode, flat extraction portion, the defect area of representing for the defect information of benchmark image; When the periphery at defect area has the pixel of scheduled volume or the extraction of the edge more than the scheduled volume, this defect area is not extracted out as the flat zone.

According to the 79th mode of the present invention; Preferably; In the image processing apparatus of the 78th mode; Flat extraction portion when the pixel of in the pixel in certain zone, being extracted out by the edge at the pixel periphery of defect area surpasses half, does not extract this defect area out as the subregional pixel in par.

According to the 80th mode of the present invention, but the computer program product that a kind of computer reads in has the image processing program that is used for requiring in the computer enforcement of rights function of any one image processing apparatus of 1～79.

Description of drawings

Fig. 1 is the figure of formation of the electronic camera of expression lens changeable formula.

Fig. 2 is the block diagram of expression electronic camera and the figure of PC (PC) and peripheral device.

Fig. 3 is the figure that the shooting to the electronic camera side in first execution mode describes in proper order.

Fig. 4 is the appearance of local standardization is carried out in expression to lightness plane figure.

Fig. 5 is the histogram of expression transmissivity mapping.

Fig. 6 is the flow chart of the flow process of the processing undertaken by PC in expression first execution mode.

Fig. 7 is the figure that the shooting to the electronic camera side in second execution mode describes in proper order.

The figure of the appearance of the change in location of dust image when Fig. 8 is the pupil location variation.

Fig. 9 is the figure of expression appearance of the size variation of dust image during as the variation of the F value of f-number.

Figure 10 is the figure of expression with respect to the one-dimensional filtering coefficient of each f-number.

Figure 11 is the figure that the filter of the transmissivity mapping that is transformed to f-number F16 is represented with the two-dimensional filtering coefficient.

Figure 12 is the flow chart of the handling process of being undertaken by PC in expression second execution mode.

Figure 13 is that expression comes the figure of the appearance of conversion transmissivity to moderate dust through the F value transform.

Figure 14 is the figure that the shooting to the electronic camera side in the 3rd execution mode describes in proper order.

Figure 15 is the figure of the flow process of the processing undertaken by PC in expression the 3rd execution mode.

Figure 16 is the figure of expression edge extraction filter.

Figure 17 is the figure of the flow process of the processing undertaken by PC in expression the 4th execution mode.

Figure 18 is expression provides the appearance of program through the data-signal of recording medium such as CD-ROM or internet etc. figure.

Figure 19 is the figure of the appearance of the periphery assimilation processing of expression edge map.

Embodiment

[first execution mode]

(formation of electronic camera and PC)

Fig. 1 is the figure of formation of single antielectron camera (below be called electronic camera) of expression lens changeable formula.Electronic camera 1 has camera main-body 2 and changes the variable optical system 3 that camera lens constitutes by assembled.Variable optical system 3 has camera lens 4 and aperture 5 in inside.Camera lens 4 is made up of a plurality of optical lens groups, but representes with a camera lens as representative in the drawings, and the position of this camera lens 4 is called key light pupil position (being designated hereinafter simply as pupil location).Variable optical system 3 also can be zoom lens.Pupil location is the value by the zoom position decision of camera lens kind, zoom lens.Sometimes change according to the focal position.

Camera main-body 2 has optics 7, imaging apparatuss 8 such as shutter 6, optical light filter or cloche.Variable optical system 3 can be with respect to department of assembly's 9 loading and unloading of camera main-body 2.In addition, variable optical system 3 will send to the control part 17 (Fig. 2) of electronic camera 1 about the optical parametrics such as information of pupil location, aperture position via department of assembly 9.F-number for example changes in F2.8～F22.

Label 10 expressions are adsorbed on the dust on optics 7 surfaces of imaging apparatus 8 front portions.Experimentize, the f-number of variable optical system 3 or pupil location are changed, estimate the variation that is written to the dust image in the photographic images, the result judges following two facts.

(1) according to f-number, the size of dust image and optical transmission rate change.

(2) according to the pupil location of camera lens, the dust offset.

Can know from these two experimental facts: even be adsorbed on the dust of fixed position, when the shooting condition (f-number and pupil location) of camera lens changed, the wiring method of dust also changed.Below expression removes the method for dust influence to this variable optical system.

Fig. 2 is the block diagram of expression electronic camera 1 and the figure of PC (PC) 31 and peripheral device.PC 31 has the function as image processing apparatus, obtains view data from electronic camera 1, and the effect of dust of stating after carrying out is removed and handled.

Electronic camera 1 has variable optical system 3, optics 7, shutter 6 (in Fig. 2, omitting diagram), imaging apparatus 8, analog portion 12, A/D converter section 13, SECO portion 14, image processing part 15, operating portion 16, control part 17, memory 18, compression/extension portion 19, display image generation portion 20, monitor 21, storage card with interface portion 22 and external interface portion 23.

Imaging apparatus 8 is taken the body that is taken through variable optical system 3, output and the corresponding picture signal (image pickup signal) of the body that is taken of taking.Imaging apparatus 8 has the camera watch region of the rectangular shape that is made up of a plurality of pixels, with as with each pixel in the picture signal of the corresponding analog signal of charge stored, be that unit outputs to analog portion 12 successively with the pixel.Imaging apparatus 8 for example is made up of one-board colored CCD etc.Analog portion 12, portion has CDS (correlated-double-sampling) circuit, AGC (automatic gain control) circuit etc. within it, to the picture signal simulation process of being scheduled to of input.A/D converter section 13 will be a digital signal by the analog signal conversion that analog portion 12 handles.SECO portion 14 is by control part 17 controls, the sequential of each action of control imaging apparatus 8, analog portion 12, A/D converter section 13, image processing part 15.

Storage card adopts the interface with storage card (removable memory of card shape) 30 with interface portion 22.External interface portion 23 adopts via the predetermined cable or the interface of external device (ED)s such as wireless transmission path and PC31.Operating portion 16 is equivalent to selector button that release-push or mode switch use etc.Monitor 21 shows various menus, or shows based on the volume image of being taken by imaging apparatus 8 or be stored in the reproduced picture of the view data in the storage card of being taken.The output of operating portion 16 is connected with control part 17, and the output of display image generation portion 20 is connected with monitor 21.Image processing part 15 for example is made up of the special-purpose single-chip microprocessor of image processing.

A/D converter section 13, image processing part 15, control part 17, memory 18, compression/extension portion 19, display image generation portion 20, storage card connect mutually via bus 24 with interface portion 22, external interface portion 23.

On PC 31, connect monitor 32 and printer 33 etc., and installation is recorded in the application program on the CD-ROM 34 in advance.In addition; PC 31; Except having not shown CPU, memory, hard disk, the storage card that also has conduct and an interface of storage card 30 is with interface portion (not shown), as the external interface portion (not shown) via the interface of predetermined cable or external device (ED)s such as wireless transmission path and electronic camera 1.

In the electronic camera 1 of the formation of Fig. 1, after being selected screening-mode and pushed release-push by the operator through operating portion 16, control part 17 carries out SECO through SECO portion 14 to imaging apparatus 8, analog portion 12, A/D converter section 13.Imaging apparatus 8 generates with passing through variable optical system 3 in the corresponding picture signal of optical imagery of camera watch region imaging.This picture signal, the analog of being scheduled to by analog portion 12, and export A/D converter section 13 to as picture signal after the simulation process.In A/D converter section 13,, and be supplied to image processing part 15 as view data with the picture signal digitlization after the simulation process.

In the electronic camera 1 of this execution mode; In imaging apparatus 8; Situation with the chromatic filter of the most representative R (red) of two (ベ イ ア) assortment one-board camera imaging apparatus, G (green), B (green grass or young crops) is an example, and the view data that is supplied to image processing part 15 is represented with the RGB colour system.In each pixel of composing images data, there is the color information of any one colour component of RGB.At this, a photo-electric conversion element that constitutes imaging apparatus 8 is called pixel, 1 unit of the view data that list is corresponding with this pixel also is pixel.In addition, image also is the notion that is made up of pixel.

Image processing part 15 carries out image processing such as interpolation, greyscale transformation, profile enhancing to this view data.Carried out the view data of this image processing, as required, the processed compressed of being scheduled to by compression/extension portion 19, and record on the storage card 30 with interface portion 22 via storage card.Carried out the view data of image processing, and also can not carry out processed compressed and just record on the storage card 30.

Carried out the view data of image processing, be provided to PC 31 via storage card 30.Also can provide to PC 31 via external interface 23 and predetermined cable or radiolink.Carried out the view data of image processing, the interpolation processing finishes, and in each pixel, has the color information of whole colour component of RGB.

(effect of dust is removed and is handled)

Next, the processing of in each view data of taking, removing effect of dust is described.In the first embodiment, suppose at every turn by 1 pair of electronic camera and be used for obtaining the situation that the benchmark image of dust information is taken according to each optical pickup condition.Yet benchmark image is not complete uniformly white reference image, can take sky, approximate uniform wall, grayscale map, monochromatic paper etc. yet and substitute.The reference data of this moment also can comprise the gray scale of the limb darkening of camera lens, the body that is taken, the shading value of imaging apparatus etc.Benchmark image is supposed the situation that can obtain under the captured situation at one's side field in fact can be easily, does not require tight uniformity, and the algorithm through the image processing side is transformed to uniformly.

(operation of electronic camera side)

Fig. 3 is the figure that the shooting to electronic camera 1 side describes in proper order.1) generally takes 101 with pupil location P1, f-number A1, export the first calibration object view data.2) then, carry out even face with identical pupil location P1, f-number A1 and take 102, export the first benchmark image data.3) then, generally take 103, export the second calibration object view data with different pupil location P2, f-number A2.4) next, take 104, export the second benchmark image data to carry out even face with general shooting 103 identical pupil location P2, f-number A2.That is, at first, electronic camera 1 is taken (the general shooting) towards the body that is taken that will take, afterwards electronic camera 1 is carried out the shooting (evenly face is taken) of even face towards sky or wall.Perhaps, camera with general take the time under the identical state, only before camera lens on the cover of about several centimetres～10 centimetres place blank sheet of paper or monochromatic paper get final product.So, carry out taking as a pair of shooting operation with generally taking with even face.At this, output image data is meant, records on the storage card 30, or directly outputs on the PC 31 via external interface 23.

The state of dust might change in electronic camera, therefore in this execution mode, after the shooting of calibration object image, directly carries out the shooting of even face with identical optical condition.But, in fact also can be so tight temporal after.Under situation about can reproduce optically for the identical shooting condition of pupil location and f-number, even the even face data through taking in about a day, considerable dust scarcely can great changes will take place can use.Therefore, evenly the face data get if take in the time difference of the degree of the dust information in the time can reproducing identical shooting condition, the general shooting of abundant reflection, just can substitute.In addition, also can exchange the order of general shooting and even face shooting, carry out even face shooting earlier and generally take then.

(operation of image processing apparatus side)

By electronic camera 1 shot image data, after having carried out the predetermined picture processing, be provided to PC 31.In PC 31, use a pair of calibration object view data and benchmark image data, carry out effect of dust and remove processing.PC 31 also can be called the image processing apparatus that carries out effect of dust removal processing.Benchmark image data and calibration object view data are handled under the state that has finished in the RGB of two assortments interpolation, all are imported among the PC 31.Below the benchmark image data and the calibration object view data of explanation are the data of under the optical condition of identical pupil location and f-number, taking.The flow chart of the flow process of the processing that Fig. 6 is undertaken by PC 31 for expression.

< to the processing of benchmark image data >

1) generation of lightness plane

In the step S11 of Fig. 6, carry out the generation of lightness plane.For each pixel [i, j] of benchmark image data, use following formula (1), generate luminance signal from rgb signal.The position of [i, j] remarked pixel.

Y[i，j]＝(R[i，j]+2*G[i，j]+B[i，j])/4……(1)

Also can individually resolve, but the influence of the dust shadow decay that only produces signal basically is irrelevant with colour component at each face of RGB.Therefore, use all information effectively, carry out conversion to the brightness composition of the influence that can reduce random noise at this.In addition, only accomplish from RGB 3 faces thus, realize high speed through the parsing of brightness composition single face.The brightness composition generates ratio and is not limited to above-mentioned ratio, also can be R: G: B=0.3: 0.6: 0.1 etc.

2) generation (extraction gain map) of transmissivity mapping

In step S12, the generation (extraction gain map) that the transmissivity of carrying out being made up of following processing is shone upon.

2-1) the standardization of part (gain is extracted out and handled)

The benchmark image data as stated may not be even fully.Therefore, the lightness plane of generation also not exclusively evenly.To this lightness plane, carry out the standardization (normalization) of local pixel value and handle, use following formula (2) to calculate the transmittance signal T [i, j] of each pixel.That is comparing, to the pixel average of each pixel value that obtains object pixel [i, j] and the subrange that comprises this pixel.Thus, the inhomogeneities of the gray scale that even face data are comprised, shading value etc. is utilized algorithm to be got rid of by no problem ground, and can only be extracted the reduction of the transmissivity that is caused by main dust shadow out.The comprehensive transmissivity of the image of so trying to achieve is called transmissivity mapping (gain map).The defect information of transmissivity mapping expression benchmark image.In addition, pixel value is the chrominance signal (color information) of the colour component in each pixel, the value of luminance signal (monochrome information).For example, during by 1 byte representation, choose 1～255 value.

$T[i,j]=\frac{Y[i,j]}{\frac{\left(\underset{m=i-a}{\overset{i+a}{\mathrm{\&Sigma;}}}\underset{m=j-b}{\overset{j+b}{\mathrm{\&Sigma;}}}Y[i+m,j+n]\right)}{(2a+1)(2b+1)}}\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\left(2\right)$

At this, choose scope (2a+1) * (2b+1) pixel of local average, adopt bigger than dust diameter.It is desirable on the area with the dust shade than the scope of selecting for use more than 3 times or 3 times, so just can obtain correct transmisivity data.A representes with the pixel count of object pixel [i, j] for expanding about the center, and b representes the pixel count expanded up and down for the center with object pixel [i, j].For example, the pel spacing of imaging apparatus 8 is made as 12 μ m, and the distance of shooting face and dust adsorption plane is made as 1.5mm, and then during f-number F22, the diameter of huge dust is about 15 pixels, and during f-number F4, the diameter of huge dust is about 40 pixels.Therefore, establish a=40, b=40, the scope of choosing local average is set at 81 * 81 pixel coverages and gets final product.This is an one of which example, also can be the pixel coverage that other pixel count is selected for use.

The dust shadow exists with ... f-number greatly, and little dust will disappear when opening aperture at once, even big dust is opened aperture, accounts for very big area though its shadow is thin out.Though also relevant with the pel spacing width of imaging apparatus, can stride tens of pixels and obtain circular dust shadow opening side sometimes, need choose local average this moment in the scope of non-constant width.Therefore, in the time of will handling high speed, even handle with pixel representative at interval also no problem.

The processing that will compare in the range computation of this (2a+1) * (2b+1) pixel is called local standard processing (gain is extracted out and handled).The filter that also can calculate in the scope of (2a+1) * (2b+1) pixel, comparing is called gain and extracts kernel (カ one ネ Le) out.Fig. 4 is the appearance of local standard processing is carried out in expression to lightness plane figure.Fig. 4 (a) is the figure that expression is arranged in the luminance signal of certain the horizontal pixel in the lightness plane.42 expressions weaken owing to the dust luminance signal label 41 with label.Fig. 4 (b) carries out above-mentioned local standard processing and the figure that obtains to the luminance signal of Fig. 4 (a).That is, in subrange, carry out the standardization of pixel value.The transmissivity of the position that the label of label 43,44 expressions and Fig. 4 (a) is 41,42 corresponding, dust exists.So, can get rid of the inhomogeneities of gray scale that even face data comprise, shading value etc., only extract out because the reduction of the transmissivity that the dust shadow produces.Thus, learn the position of dust and the degree of transmissivity simultaneously.

2-2) low pass of transmissivity mapping is handled

Also can make the low pass of transmissivity mapping handle and to select, but, should handle so preferably get in advance because major part has effect.At transmittance signal T [i; J] in; Comprise the random noise that the fluctuation along with the quantum theory of luminance signal causes, therefore be approximately 1 the degree residual zone that the influence of dust shadow is arranged slightly, because its randomness in transmissivity; At the following 2-4 that carries out) threshold determination the time, in spot, extract the dust shadow sometimes out.For anti-thing here, according to following formula (3), when carrying out the collectivization of dust shadow through low pass filter, it is more attractive in appearance to become.

T[i，j]＝{4*T[i，j]

+2*(T[i-1，j]+T[i+1，j]+T[i，j-1]+T[i，j+1])

+1*(T[i-1，j-1]+T[i-1，j+1]+T[i+1，j-1]+T[i+1，j+1])}/16……(3)

2-3) statistics of transmissivity mapping is resolved

Image for the transmissivity mapping that obtains through above-mentioned local standard processing is comprehensive, tries to achieve mean value M according to following formula (4), tries to achieve standard deviation according to following formula (5), adds up parsing.In addition, N _x, N _yThe total pixel number of expression x direction, y direction.

$M=\frac{1}{N_x{N}_y}\underset{i,j}{\mathrm{\&Sigma;}}T[i,j]\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\left(4\right)$

$\mathrm{\&sigma;}=\sqrt{\frac{1}{N_x{N}_y}\underset{i,j}{\mathrm{\&Sigma;}}{\left(T\right[i,j]-M)}^2}\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\left(5\right)$

2-4) threshold determination

Basically the ratio of the area of shared dust signal is very little in the transmissivity mapping, 2-3) in the result that resolves of statistics, estimated the random noise (short clutter) that the fluctuation along with the quantum theory of transmittance signal causes.Amplified the label 46 of label 45 parts among Fig. 4, there is the appearance of this tiny random noise in expression.When adopting the histogram of transmissivity mapping, becoming with mean value M (M is approximately 1 value) is the shape of central standard deviations normal distribution.Fig. 5 is the histogrammic figure of expression transmissivity mapping.Therefore the influence of the variation of the transmissivity that the scope of considering this fluctuation does not receive to be caused by the dust shadow can be set at 1 with transmissivity by the strong hand.That is, carry out threshold determination according to following condition (6) (7).

If | T [i, j]-M|≤3 σ are T [i, j]=1 so ... (6)

Otherwise T [i, j]=T [i, j] ... (7)

The random data of normal distribution if concentrate on ± scope of 3 σ, will become 99.7%, influence that therefore can ballpark removal random noise.Transmissivity outside ± 3 σ is the anomalous signals that can't explain in the error of statistics almost, the phenomenon that the reduction of the transmissivity of having represented to cause owing to the dust shadow produces.This unusual part under the situation of dust shadow, generally is than 1 little value.

But, though ratio is also represented the value bigger than 1 seldom sometimes.This is not the influence of dust shadow, but is causing under the situation such as strengthening or weaken the incident Light Interference Streaks, sees the phenomenon of the defective that the striped (inequality of refractive index) by optical low-pass filter etc. produces.Thus, can also utilize this method to the defects detection of the optics beyond the dust that is comprised in the light path way.In addition, the influence of the picture element flaw in the imaging apparatus also can be differentiated through this method.Dust near imaging apparatus 8 is apparent in view, and is residual easily, even but also can high-precisionly differentiate under the quite fuzzy situation about writing of the dust on the taking lens.

In addition, only remove under the situation of influence of dust shadow, carry out threshold determination according to following condition (8) (9) (10) and get final product.

If | T [i, j]-M|≤3 σ are T [i, j]=1 so ... (8)

T [i, j]＞1 T [i, j]=1 else if ... (9)

Otherwise T [i, j]=T [i, j] ... (10)

Because the mean value M that is used to judge selects for use usually and is approximately 1 value, so also can replace with 1.

Obtain representing two kinds of defect information of degree of figure information (so that whether T=1 judges) and expression defective of the location of pixels of defective so, simultaneously.In addition, therefore the local relative gain of above-mentioned transmissivity mapping expression also can be called gain map.

Usually, the defects detection of dust etc., the differential filter of being used by rim detection carries out.But, with the dust in the light path way during as object, since fuzzyyer on the optics, therefore be rendered as the low-down dust shadow of contrast with periphery.In this case, mostly at the non-constant of differential filter medium sensitivity, almost can't detect.But, as above state brightly, if use the criterion of the statistical property utilized transmissivity, just can carry out the very high dust of sensitivity and detect, can to since the influence that the foreign matter of the light path of purpose on the way causes proofread and correct.

< to the calibration object treatment of picture >

3) gain calibration

In step S13, carry out gain calibration.Utilize the transmissivity mapping of as above trying to achieve to carry out the correction of calibration object view data.Shown in (11) (12) (13), to R, G, the B of the calibration object view data inverse with transmittance signal on duty, carry out gain calibration respectively.

R[i，j]＝R[i.j]/T[i，j]……(11)

G[i，j]＝G[i.j]/T[i，j]……(12)

B[i，j]＝B[i.j]/T[i，j]……(13)

Thus, can proofread and correct well because the reduction of the brightness that the dust shadow causes.In addition, because threshold determination is carried out in mapping to transmissivity, unnecessary correction can not be carried out in the place that therefore need not to proofread and correct.That is,, be not exaggerated so worry the clutter of rgb signal because the transmissivity T of the position that do not have dust has been removed the influence of random noise.

As above, in the first embodiment,, also can carry out suitable correction to the image of taking at any time even do not have the common electronic camera of the special organization that the dust countermeasure uses.Because the shooting to the even face of benchmark image does not require tight uniformity, therefore can realize with comparalive ease.And then, compare with existing dust detection method, can carry out sensitivity good especially detection and correction.

[second execution mode]

In second execution mode, represented following method: only take the benchmark image that once is used to obtain dust information, even a plurality of images different to the optical pickup condition also utilize this benchmark image to remove dust.Electronic camera 1 and as the structure of the PC 31 of image processing apparatus, identical with first execution mode, therefore omit its explanation.

(operation of electronic camera side)

Fig. 7 is the figure that the shooting to electronic camera 1 side in second execution mode describes in proper order.1) carries out even face with pupil location P0, f-number A0 and take 201, the output reference view data.2) generally take 202 with pupil location P1, f-number A1, export the first calibration object view data.3) generally take 203 with pupil location P2, f-number A2, export the second calibration object view data.4) generally take 204 with pupil location P3, f-number A3, export the 3rd calibration object view data.That is, at first electronic camera 1 is carried out the shooting (evenly face is taken) of even face towards sky or wall, thereafter, electronic camera 1 is taken (the general shooting) at any time towards the body that is taken that will take.

At this, the f-number A0 of benchmark image is in the variable range that variable optical system 3 is prepared, to turn to the f-number that minimum state is taken.Turning to minimum f-number, for example is F22 in the camera lens of standard.On the other hand, make the f-number of calibration object image identical with benchmark image or than its more open side.

Evenly face is taken as long as the adsorbed state of dust is constant and just can be omitted.Evenly the insertion number of times taken of face do not surpass a lot, but usually if the data of degree once a day just can obtain effective dust data.Whether carry out the judgement that even face is taken, undertaken by photographer oneself.But the even face that before carries out is taken under the situation too of a specified duration at interval in time, also can produce the situation of the reference data shortcoming reliability of taking based on this even face.Therefore, also only service range generally take the benchmark image data that the even face in preset time is taken.When before and after general the shooting, existing a plurality of even faces to take, the benchmark image data that can use nearest in time even face to take.Perhaps, if in the possibility that new absorption dust is arranged, also can before and after taking, choose one of which wantonly the second nearly shooting before.

(operation of image processing apparatus side)

In second execution mode, in being input to as benchmark image data and calibration object view data among the PC 31 of image processing apparatus, embedding can be discerned the data of pupil location and f-number.The pupil location data also can be utilized conversion table, and the record data of the kind of the camera lens from be embedded into photographed data, zoom position, focal position calculate.Figure 12 is the flow chart of the flow process of the processing undertaken by PC 31 of expression.

< to the processing of benchmark image >

1) generation that lightness plane generates, transmissivity is shone upon

The lightness plane of step S21 generates, the generation of the transmissivity of step S22 mapping, carries out with first execution mode the samely.

2) the pupil location conversion of transmissivity mapping

In step S23, carry out the pupil location conversion of transmissivity mapping.The pupil location of benchmark image and calibration object image each other not simultaneously, conversion pupil location, the dust position that occurs when the dust evolution of benchmark image is seen for the pupil location from the calibration object image of prediction.The figure of the appearance of the change in location of dust shadow when Fig. 8 changes for the expression pupil location.Fig. 8 (a) is the figure of the shooting relation of plane of expression pupil location, dust and imaging apparatus 8.Fig. 8 (b) is the figure of expression along with variation dust shadow mobile appearance on shooting face of pupil location.

Can know from Fig. 8, pupil location not simultaneously, the position that is written to the dust in the image is to squint in the radius vector direction in the center of image from optical axis 51.At this, the optical axis of estimating to be positioned at range image is the amount Δ r that the dust of the position of r squints on the radius vector direction.If the pupil location of benchmark image is P0, the pupil location of calibration object image is P0 ', and it is the position of l that dust is adsorbed on apart from shooting face, and then Δ r can calculate according to following formula (14).

$\mathrm{\&Delta;r}=r\&CenterDot;\frac{\mathrm{<figure-callout\; id="0"\; label="l\; P"\; filenames="HSA00000036066700051.png,HSA00000036066700111.png"\; state="\{\{state\}\}">l</figure-callout>}}{P_{}^{}}\&CenterDot;\frac{P_0-{\mathrm{<figure-callout\; id="0"\; label="P"\; filenames="HSA00000036066700051.png,HSA00000036066700111.png"\; state="\{\{state\}\}">P</figure-callout>}}_0^{\&prime;}}{{\mathrm{<figure-callout\; id="0"\; label="P"\; filenames="HSA00000036066700051.png,HSA00000036066700111.png"\; state="\{\{state\}\}">P</figure-callout>}}_0}\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\left(14\right)$

Wherein, be the value that the thickness conversion of optics has been become airborne optical path length apart from l.

With the transmissivity of benchmark image mapping T [i, j] according to following formula (15) at polar coordinates [r, θ] superior displacement to [r ', θ], thereby be transformed to the transmissivity mapping T ' [i, j] on the coordinate [i, j].

$r^{\&prime;}=r+\mathrm{\&Delta;r}=r(1+\frac{\mathrm{<figure-callout\; id="0"\; label="l\; P"\; filenames="HSA00000036066700051.png,HSA00000036066700111.png"\; state="\{\{state\}\}">l</figure-callout>}}{P_{}^{}}\&CenterDot;\frac{P_0-{\mathrm{<figure-callout\; id="0"\; label="P"\; filenames="HSA00000036066700051.png,HSA00000036066700111.png"\; state="\{\{state\}\}">P</figure-callout>}}_0^{\&prime;}}{P_0})\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\left(15\right)$

Along with big with the change of the distance of optical axis 51, it is big that offset r becomes.At the peripheral part of the image of reality, according to the value of pupil location, sometimes even reach tens of pixels.

3) the F value transform of transmissivity mapping

In step S24, carry out the F value transform of transmissivity mapping.The f-number of benchmark image and calibration object image mutually not simultaneously, with the dust diameter of benchmark image and transmissivity F value transform in dust diameter and the transmissivity of calibration object image under the f-number of more opening a side.Fig. 9 is the figure of the appearance of the size variation of dust shadow during as the variation of the F value of f-number for expression.The bigger situation of Fig. 9 (a) expression F, the less situation of Fig. 9 (b) expression F.Can know from Fig. 9, when the definition (effective aperture of F=focal length/camera lens) with the F value be applied to have similarity relation from shooting face to the dust absorption position apart from l and dust scope Г the time, following formula (16) is set up.

$\mathrm{\&Gamma;}=\frac{l}{F}\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\left(16\right)$

When l was removed by the pel spacing a [mm/pixel] of imaging apparatus, the dust diameter can be represented with pixel count.So, can predict when aperture is the F value that the dust of some picture is amplified to the size of width Г.

On the other hand and since consider from each incident angle of the camera lens of f-number inner opening equably to point as dust irradiates light expansion dust shadow, so the distribution function of this point picture is assumed to and has the function of expanding fully uniformly and get final product.Therefore,, handle, can correctly predict the F value transform of dust diameter and transmissivity through the homogeneous low-pass filter device that carries out representing by filter width Г pixel for the F value transform.Low pass filter is that diameter is the common circular non-divergence type filter of Г, but the high speed in order to handle also can be the foursquare divergence type filter of vertical Г, horizontal Г.

For example, when being applied to when l=0.5mm, a=5 μ m/pixel transmissivity mapping transformation with F22 and being the situation of F16, F11, F8, F5.6, F4, the one-dimensional filtering coefficient of foursquare divergence type filter is represented as the form of Figure 10.Use the one-dimensional filtering coefficient of Figure 10, carry out filtering in length and breadth respectively.In addition, to have two ends be 7 coefficients of 0.5 to the one-dimensional filtering coefficient of f-number F16.This is because be the center with the object pixel, about the impartial up and down expansion wide degree of even number and carry out filtering with the wide scope of odd number.Figure 11 is for representing the figure of f-number F16 filter with two dimensional filter.

Through carrying out above-mentioned conversion process, the transmissivity of state that the mapping of the transmissivity of benchmark image is transformed to pupil location, the F value of calibration object image is shone upon.That is, the mapping of the transmissivity of benchmark image generates the transmissivity mapping with the transmissivity mapping equivalence that under the optical condition of taking the calibration object image, generates.

< to the calibration object treatment of picture >

3) gain calibration

In step S25, utilize above-mentioned conversion process transmissivity mapping carry out gain calibration.Identical with first execution mode, respectively to R, G, the B value of calibration object view data, shown in (17) (18) (19), multiply by the inverse of the transmittance signal behind the F value transform, carry out gain calibration.

R[i，j]＝R[i.j]/T’[i，j]……(17)

G[i，j]＝G[i.j]/T’[i，j]……(18)

B[i，j]＝B[i.j]/T’[i，j]……(19)

Figure 13 is that expression comes the figure of the appearance of conversion transmissivity to moderate dust through the F value transform.Transverse axis remarked pixel position, the longitudinal axis is represented transmissivity.

So, can be in variable optical system, only take a benchmark image, and do not need the shooting of the benchmark image under other the optical condition with the minimum aperture value.That is, can through with benchmark image conversion dust data reciprocally, realize effectively proofreading and correct.Therefore, identical with first execution mode, do not need the tightness of even image taking, also can keep the detection performance of high sensitivity.

[the 3rd execution mode]

In the 3rd execution mode, be illustrated under the state of the benchmark image that does not have even face fully, carry out in the calibration object image that dust detects and with the method for its removal.Basic principle is following: if find the flat (image section is the zone uniformly) in the calibration object image, application that can be directly identical generates processing (gain map generation) to the dust transmissivity mapping that first benchmark image implemented carries out.Therefore electronic camera 1 and identical with first execution mode as the structure of the PC 31 of image processing apparatus omits its explanation.

(operation of electronic camera side)

Figure 14 is the figure that the shooting to electronic camera 1 side in the 3rd execution mode describes in proper order.1) generally takes 301 with pupil location P1, f-number A1, export the first calibration object view data.2) generally take 302 with pupil location P2, f-number A2, export the second calibration object view data.3) generally take 303 with pupil location P3, f-number A3, export the 3rd calibration object view data.4) generally take 304 with pupil location P4, f-number A4, export the 3rd calibration object view data.That is, in the 3rd execution mode, the even face that does not carry out carrying out in first execution mode and second execution mode is taken.

(operation of image processing apparatus side)

< to the calibration object treatment of picture >

Figure 15 is the flow chart of the processing carried out among the PC 31 that is illustrated in as image processing apparatus.In step S31, carry out the generation of lightness plane.In step S32, extract filter process, threshold determination out through γ correction, the edge of lightness plane, carry out the generation of edge map.In step S33, edge map carried out black dull the processing of appending.In step S34, carry out the processing and amplifying of edge map.In step S35, proceed to the conversion of smooth mapping.In step S36, carry out oneself's gain and extract processing out.In step S37, carry out self-gain calibration and handle.Below, each step is elaborated.

1) generation of lightness plane (step S31)

To each pixel [i, j] of calibration object view data, carry out the conversion of rgb signal to brightness signal Y.Transform method is identical with the method that benchmark image to first execution mode carries out.

2) generation of edge map (step S32)

Extract the edge out filter and be placed on the lightness plane, carry out separating of interior flat of calibration object image and marginal portion.The dust mapping that is comprised in the light path way is rendered as the low-down dust shadow of contrast as interior writing, and it is many therefore to extract the situation that filter detection do not go out out through the edge of prior art.Utilizing on the contrary should the fact, extracts the marginal portion that filter is extracted out out by the edge, if not dust but the edge in the basic image then can be assumed to a plurality of positions.In order further to carry out edge and the difference of dust in this image, at first carry out gradation correction processing to lightness plane.

2-1) γ of lightness plane proofreaies and correct

Linear gray scale is directly imported the calibration object image, generates above-mentioned lightness plane.At this moment, (0≤Y≤Ymax), output signal are Y ' (0≤Y '≤Y ' max), carry out the greyscale transformation as shown in the formula (20) to establish input signal and be Y.In addition, γ=0.4,0.5,0.6 etc.

$Y^{\&prime;}=Y_{\max }^{\&prime;}{\left(\frac{Y}{Y_{\max }}\right)}^{\mathrm{\&gamma;}}\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\left(20\right)$

This be transformed to the semi-tone of rising low-light level side contrast, reduce the processing of the contrast of high brightness side.That is, very distinct owing to be difficult to see clearly the dust shadow, so the contrast of dust shadow reduces through this conversion, and because general image border mainly is distributed as semi-tone at bright local dust shadow in dark place, therefore relative rising contrast.Therefore, the dust shadow improves with the separating degree of the contrast at general edge.And then will result from the short clutter of quanta fluctuation of Y ' when carrying out conversion through whole uniform gray level with handling, consider to choose γ=0.5 the best from the error propagation rule.In addition, above-mentioned formula (20) becomes power function.In addition, γ=0.5, power function becomes square root function.

When input picture is finally exported the γ treatment for correcting of usefulness, under the situation of carrying out with the approaching γ treatment for correcting of above-mentioned conversion, also can skip this processing.In addition, carry out contrary γ proofread and correct return linear gray scale after, when carrying out above-mentioned processing, obtain better separation function.

2-2) filter process is extracted at the edge out

Then, on the lightness plane of carrying out the γ correction, put the edge of Figure 16 and following formula (21) and extract filter out.If it is YH [i, j] that composition is extracted at the edge of each pixel out.

YH[i，j]＝{|Y’[i-1，j]-Y’[i，j]|+|Y’[i+1，j]-Y’[i，j]|

+|Y’[i，j-1]-Y’[i，j]|+|Y’[i，j+1]-Y’[i，j]|

+|Y’[i-1，j-1]-Y’[i，j]|+|Y’[i+1，j+1]-Y’[i，j]|

+|Y’[i-1，j+1]-Y’[i，j]|+|Y’[i+1，j-1]-Y’[i，j]|

+|Y’[i-2，j-1]-Y’[i，j]|+|Y’[i+2，j+1]-Y’[i，j]|

+|Y’[i-2，j+1]-Y’[i，j]|+|Y’[i+2，j-1]-Y’[i，j]|

+|Y’[i-1，j-2]-Y’[i，j]|+|Y’[i+1，j+2]-Y’[i，j]|

+|Y’[i-1，j+2]-Y’[i，j]|+|Y’[i+1，j-2]-Y’[i，j]|

+|Y’[i-3，j]-Y’[i，j]|+|Y’[i+3，j]-Y’[i，j]|

+|Y’[i，j-3]-Y’[i，j]|+|Y’[i，j+3]-Y’[i，j]|

+|Y’[i-3，j-3]-Y’[i，j]|+|Y’[i+3，j+3]-Y’[i，j]|

+|Y’[i-3，j+3]-Y’[i，j]|+|Y’[i+3，j-3]-Y’[i，j]|}/24……(21)

At this, be: the original marginal portion of extraction image of trying one's best complete, the concentrated absolute value difference of not omitting from all directions with a plurality of correlation distances with above-mentioned Design of Filter.

2-3) threshold determination

According to following formula (22) (23), composition YH is extracted out in the edge carry out threshold determination, carry out the classification of marginal portion or flat, and the result is outputed to edge map EDGE [i, j].Threshold value Th1 selects the value about 1～5 for use to 255 gray scales.The dust shadow that exists on the marginal portion, the dust shadow is hidden in the signal of marginal portion vibration basically, and not obvious, therefore the zone for need not to remove.

If YH [i, j]＞Th1 EDGE [i, j]=1 (marginal portion) ... (22)

Otherwise EDGE [i, j]=0 (flat) ... (23)

As stated, at 2-1) in carry out greyscale transformation, change the weighting between gray scale, at 2-3) in carry out threshold determination through whole gray scales and in the threshold value Th1 that fixes.But, carry out threshold determination even directly carry out edge extraction, setting and brightness degree corresponding threshold with linear gray scale, also can obtain roughly the same effect.

3) append black dull (step S33) in the mapping on the edge of

Edge map is represented to carry out the zone that gain map is extracted out.Except the edge, the zone of carrying out gain map extraction hazardous is dark zone (black dull portion).Black dull because S/N is very poor, even therefore extraction relative gain reliability is also very low.And then, therefore need not to carry out dust and remove because black dull the dust shadow of going up existence almost can't see.Therefore, dark zone also is appended in the edge map according to following formula (24).Threshold value Th2 corresponding 255 linear gray scales approximately are set in below 20 or 20.Schematically this operation table is shown " EDGE '=EDGE+DARK " will understand easily.

If Y [i, j]≤Th2 EDGE [i, j]=1 ... (24)

4) processing and amplifying of edge map (step S34)

Identical with first execution mode, central value and comparing of mean value with (2a+1) * (2b+1) pixel in flat compare, and generate the transmissivity mapping.Therefore, carry out (2a+1) * (2b+1) pixel processing and amplifying of marginal portion according to following formula (25), in advance this kernel is not put in the marginal portion.m＝1、2……a，n＝1、2……b。

If EDGE [i, j]=1 EDGE [i ± m, j ± n]=1 ... (25)

5) to the conversion (step S35) of smooth mapping

According to following formula (26) (27) edge map EDGE [i, j] is transformed to smooth mapping FLAT [i, j].Realize through bit reversal.The flat site that smooth mapping is represented, the gain map that expression is made up of (2a+1) * (2b+1) pixel are extracted the zone that kernel can the oneself extracts out in the calibration object image out.

If EDGE [i, j]=0 FLAT [i, j]=1 (flat) ... (26)

Otherwise FLAT [i, j]=0 (marginal portion) ... (27)

6) (step S36) extracted in oneself's gain out

Only the zone of FLAT [i, j]=1 is carried out generating the processing sequence that transmissivity is shone upon from the benchmark image data in first execution mode.

6-1) local standard processing (gain is extracted out and handled)

T [i, j] from the zone that interior the comparing of (2a+1) * (2b+1) pixel generates FLAT [i, j]=1.FLAT [i, j]=0 zone is set at whole T [i, j].

6-2) statistics of transmissivity mapping is resolved

Identical geo-statistic with first execution mode is resolved the interior T [i, j] in zone of FLAT [i, j]=1, calculating mean value m and standard deviation.

6-3) threshold determination

T [i, j] in the zone of FLAT [i, j]=1 is carried out threshold determination with first execution mode identically, select for use the T [i, j] of the value of m ± 3 σ to be set at T [i, j]=1.

7) self-gain calibration (step S37)

Ground identical with first execution mode to the inverse of R, G, the B of the calibration object image transmittance signal T [i, j] that extracts out with the oneself on duty, carries out self-gain calibration respectively.

So, even without the benchmark image data of even face, also can the oneself extract the dust of calibration object image itself out and proofread and correct.That is, in a photographic images, extract the zone of satisfying the predetermined condition that can guarantee flatness as stated out.The same area that makes extraction be benchmark image be again the calibration object image.In addition, in the 3rd execution mode, do not need all to add the influence of variable optical system.Particularly, the removal of the little dust shadow that occupies huge quantity is exploited one's power.

[the 4th execution mode]

In the 4th execution mode; Identical with second execution mode, only take a benchmark image, utilize the information relevant with the dust position; And the transmissivity mapping is not to adopt the mode of extracting out from benchmark image, but adopts the mode of extracting out from calibration object image oneself itself like the 3rd execution mode.In second execution mode, carry out the pupil location conversion of transmissivity mapping, but the value of pupil location is an approximation when being inaccurate, be created in the situation that produces error in the pupil location conversion.On the other hand, in the 3rd execution mode, mapping is extracted big dust out in extracting out as the edge on the edge of, produces the situation of not proofreading and correct.In the 4th execution mode, the problem of this second execution mode and the 3rd execution mode is handled.That is, utilize the generation method of the transmissivity mapping that reliability is high in the 3rd execution mode, proofread and correct through the high dust positional information of reliability that obtains with the method identical simultaneously with second execution mode.In addition, electronic camera 1 and as the structure of the PC 31 of image processing apparatus, identical with first execution mode, therefore omit its explanation.

(operation of electronic camera side)

The shooting order is identical with second execution mode.

(operation of image processing apparatus side)

The flow chart of the processing that Figure 17 is undertaken by the PC 31 as image processing apparatus for expression.

< to the processing of benchmark image >

1) generation of the lightness plane of step S41 is identical with first execution mode and second execution mode.2) generation (gain map extraction) of the transmissivity of step S42 mapping is identical with first execution mode and second execution mode.3) the pupil location conversion of the transmissivity of step S43 mapping is identical with second execution mode.4) the F value transform of the transmissivity of step S44 mapping is identical with second execution mode.

5) threshold determination of transmissivity mapping

In step S45, carry out the threshold determination of transmissivity mapping.When carrying out the F value transform of transmissivity mapping, generate a lot of pixels approaching with handle transmissivity that the dust shadow almost disappears 1 through low pass filter.For these are other with distinct dust shadow zone, carry out threshold determination once again according to following formula (28) (29).At this, be utilized in 2 once again) " transmissivity mapping generation " process in the standard deviation value σ that calculates.With T ' [i, j] expression pupil location, F value transform the transmissivity mapping.

If | T ' [i, j]-1|≤3 σ are T ' [i, j]=1 so ... (28)

Otherwise T ' [i, j]=T ' [i, j] ... (29)

The conversion of 6) shining upon to dust

In step S46, according to following formula (30) (31) transmissivity is shone upon binaryzation, and be transformed to dust mapping dmap [i, j].

If T ' [i, j]＜1 dmap [i, j]=1 ... (30)

Otherwise dmap [i, j]=0 ... (31)

Judgement in this formula (30) also can make it have some leeway, judges like T ' [i, j]＜0.95.

7) processing and amplifying of dust mapping

In step S47, through only amplifying the dust mapping of the error part of in the pupil location conversion, supposing according to following formula (32), making the zone in the admissible error is the dust mapping that comprises dust.At this, for example estimate the error of ± 3 pixels.m＝1、2、3，n＝1、2、3。

If dmap [i, j]=1 dmap [i ± m, j ± n]=1 ... (32)

< to the calibration object treatment of picture >

1) generation of the lightness plane of step S51, identical with the 3rd execution mode.2) generation of the edge map of step S52 is identical with the 3rd execution mode.3) edge map of step S53 is appended black dull processing, identical with the 3rd execution mode.Schematically this operation table is shown " EDGE '=EDGE+DARK " will understand easily.

4) from edge map, remove the dust zone

In step S54, remove the dust zone from edge map.The majority of dust shadow is not extracted out by the edge because contrast is very low, wherein has dust big, that contrast is high, will be extracted out by the edge.Particularly in rotating the calibration object image of taking, in some dust shadows, taken place.Depart from order to prevent that these dust shadows from extracting out the regional appointment from gain as fringe region, carry out following processing: be utilized in the dust map information of trying to achieve among the step S46, remove the dust position by the strong hand from the marginal portion according to following formula (33).At this, in order to prevent to remove greatly fringe region, utilize before the processing and amplifying of the dust mapping of carrying out step S47 the dust mapping.Schematically this operation table is shown " EDGE "=EDGE+DARK-DUST " will understand easily.

If dmap [i, j]=1 EDGE [i, j]=0 ... (33)

4 ') assimilation of the periphery of edge map is handled (correction that dust digs out part) (S60)

Edge map is only dug out (removal) dust part by the shifting ground state is factitious, and the periphery assimilation of therefore carrying out in the edge map is handled.For example, when background is the even image of sky and so on, utilize the dust map information of step S46, any problem can not take place in the marginal portion of from edge map, digging out big dust yet.Would be better and to dig out.But background is to have under the situation of image of pattern and structure, according to the dust map information of step S46 and when from edge map, digging out the part that has dust, carries out factitious treatment for correcting from the pattern of the reality of periphery and the relation of structure.Therefore, be judged as the pixel that is not the marginal portion, by being judged when in its peripheral pixel, having a lot of edge pixel, with this pixel as marginal portion once again.

The periphery assimilation processing of edge map is carried out according to the processing shown in following.Specifically; Periphery 8 pixels relative with object pixel for example shown in Figure 19 (pixel of the bullet of Figure 19, Figure 19 is to object pixel [i, j]=[0; 0] only represent four-quadrant) in to surpass 4 pixel be under the situation at edge, make object pixel also be the edge.Surpass 4 and be meant in peripheral 8 pixels it is that the pixel at edge surpasses half.That is, in neighboring pixel, exist under the situation of the edge pixel that surpasses half, making object pixel also is the edge.In addition, in the example of following processing,, may not necessarily be defined as the 8th pixel though see the pixel of the 8th pixel of horizontal direction vertical direction.See that from several pixels the pixel before tens pixels gets final product.In addition, following processing also can be carried out whole pixels, also can only limit to the pixel that the edge digs out with dmap=1.

Data are duplicated

Tmp [i, j]=EDGE [i, j] is for all pixels [i, j]

The periphery assimilation is handled

If tmp [i, j]=0{

sum＝tmp[i-8，j]+tmp[i+8，j]+tmp[i，j-8]+tmp[i，j+8]

+tmp[i-8，j-8]+tmp[i+8，j+8]+tmp[i-8，j+8]+tmp[i+8，j-8]

If sum＞4 EDGE [i, j]=1

}

5) the edge map processing and amplifying of step S55 is identical with the 3rd execution mode.6) conversion to smooth mapping of step S56 is identical with the 3rd execution mode.

7) confirming of zone extracted in oneself's gain out

In step S57, carry out oneself's gain and extract confirming of zone out.The zone that is defined as flat and confirms dust zone is carried out dust and is removed, and is the most rational from the viewpoint of the error correction that prevents the calibration object image.Therefore, bring the area information that satisfies these two conditions into try to achieve smooth mapping according to following formula (34).That is, only all be in FLAT=1 under the situation of 1 mark FLAT and dmap both sides, otherwise FLAT=0.

FLAT[i，j]＝FLAT[i，j]*dmap[i，j]……(34)

8) oneself's gain is extracted out

The oneself of the step S58 extraction that gains, different with the 3rd execution mode, only carry out local standard processing (gain is extracted out and handled).Resolve according to the statistics of the transmissivity after this mapping and to handle the dust area limiting that carries out with threshold determination and handle, owing to according to 7) processing be limited near the dust, so do not need.Local standard processing (gain is extracted out and handled) is identical with the 3rd execution mode.Like this, only extract the zone out, carry out the dust search, can not have thus and omit the and carry out dust and extract out in the dust circumferential expansion gain that the error of pupil location conversion precision is divided according to the processing of step S47.

Carrying out low pass in this transmissivity mapping that oneself is extracted out handles.The Zone Full that the oneself of T [i, j] is extracted out carries out the low pass identical with first execution mode to be handled, and removes the fluctuation composition of the pixel [i, j] that T [i, j] comprised.In this execution mode,, or not do not gain to extract out through the oneself and carry out self-gain calibration, so this low pass processing is important processing through resolving the threshold process of carrying out according to adding up only for the regional area that is decided to be the dust position.That is, before low pass was handled, because pixel value fluctuates on equidirectional with transmittance values T [i, j], during the self-gain calibration therefore not carrying out stating after the low pass processing is just carried out, image became only in this regional area overall flat easily.Therefore,, then do not lose the fluctuation composition that pixel value has, can guarantee continuity yet with the graininess of peripheral part if remove the fluctuation composition of T [i, j].This exploits one's power under the many situation of the scrambling of high sensitivity image and so on especially.At this, though ratio first execution mode of low pass filter design stronger slightly also it doesn't matter, it is also passable that the big dust part (value of T [i, j] is compared quite little place with 1) that perhaps just receives the low pass filter influence is easily missed the low pass filter processing.

9) self-gain calibration

The self-gain calibration of step S59 is identical with the 3rd execution mode.Even because the pupil location conversion deterioration in accuracy of the transmissivity of benchmark image mapping, therefore the transmissivity information that yet can extract dust from calibration object image itself out can not have the perfect correction of deviation fully.In addition, the extraction of oneself gain is limited to the oneself who confirms among the step S57 and gains and extract the zone out and carry out.Therefore, treatment for correcting also only limits to this scope carries out, and has alleviated processing burden.

As above, in the 4th execution mode, through effective the utilize benchmark image the dust map information, can not have and omit the oneself and extract all dusts in the calibration object image out from big dust to little dust.In addition, during the pupil location conversion low precision of the transmissivity of benchmark image mapping, can be as the replacement equipment of second execution mode.And then identical with second execution mode, the photographer's that benchmark image is taken burden is also very little.

In the image processing apparatus of first execution mode～the 4th execution mode of above-mentioned explanation; The defective such as black of dying that can cause the influence by dust etc. that in the image that is being used arbitrarily constantly by electronic camera, taking under the service condition arbitrarily, produces is suitably proofreaied and correct, and reproduces high quality images.

In addition, above-mentioned first, second, in the 4th execution mode, in order to make the transmissivity mapping, photographer takes and thinks near benchmark image uniformly, and the benchmark image of this shooting is carried out local standardization, processes the transmissivity mapping.But photographer thinks near having little pattern in the benchmark image uniformly sometimes.This can handle through making the body that is taken swoon to reflect to take basically.For example, paper being placed on the position also nearer than the shortest photographing distance of camera lens takes and gets final product.Even little pattern is arranged,, will fully realize purpose, thereby can obtain near uniform benchmark image if the picture that in than the big scope of the gain extraction kernel of (2a+1) * (2b+1) size, slowly changes is thickened.

In addition, in above-mentioned the 4th execution mode, carry out oneself's gain at step S57 and extract the definite of zone out, S58 proofreaies and correct in restricted portion in step.So correcting range is defined as the method for the neighboring area (near zone) that comprises the dust zone, in above-mentioned first～the 3rd execution mode, also can be suitable for.In first～the 3rd execution mode, confirm dust from the transmissivity mapping of trying to achieve, try to achieve its neighboring area and get final product.

In addition, in above-mentioned the 3rd execution mode, the processing that obtains a photographic images and the extraction flat generates the dust mapping in a photographic images is illustrated.But, exist at flat under the situation of big dust, this part is not extracted out as flat.In the 4th execution mode, represented through obtaining benchmark image its example of handling; Even but mustn't go to benchmark image; Also can the part of this big dust also be thought the flat of defect information making object through utilizing the relevant of a plurality of images.For example, in a plurality of photographic images of the different bodies that is taken, always have when extracting detected picture out by the edge in same position, having that is the possibility of the picture of dust.Therefore, for the edge map of in the 3rd execution mode, handling, when in a plurality of photographic images, getting AND, the part of getting AND is removed from edge map.So, can the part of getting AND be appended in the flat, even also can process the transmissivity mapping big dust.That in addition, gets AND also can be not limited to edge map.From the data that photographic images generates, so long as get the data that AND can think to take the dust on the light path, what data can.For example; No matter gain extracts whether the transmissivity mapping of kernel is flat out; By the strong hand image is calculated comprehensively; Have under the situation of the picture of extracting out with identical transmissivity in identical position between a plurality of images, also can consider the method that obtains of following AND: the transmissivity as defect information is shone upon and residual, and other are removed from defect information.

In addition, for the mapping of the transmissivity in the flat that in the 3rd execution mode, obtains, when getting the OR of a plurality of photographic images, can obtain the whole transmissivity mapping of shooting picture.The flat that in the 3rd execution mode, obtains, also different according to the position of flat in the different shooting picture of the body that is taken.If get the OR of these flats, also can become shooting picture integral body.Therefore, even do not take the benchmark image that is used to obtain dust information, can be that a plurality of calibration object images obtain the whole transmissivity mapping of shooting picture also from a plurality of photographic images.The transmissivity mapping that this shooting picture is whole can be shared in a plurality of calibration object images.

In addition, when getting the AND of edge map for a plurality of photographic images as stated or getting the OR of transmissivity mapping, also different according to photographic images pupil location and F value (f-number).At this moment, with in second execution mode, explain identical, directly under the state of the state of picture signal or transmissivity mapping, carry out pupil location conversion or the conversion of F value signal etc. and apply flexibly getting final product.

In addition, in the above-described embodiment, the example of the RGB colour system of two assortments has been described, if but finally carrying out the interpolation processing, just do not exist with ... the collocation method of color filter certainly fully.In addition, other colour system (for example complementary color colour system) too.

In addition, in the above-described embodiment, the example of single antielectron camera of lens changeable formula has been described, but may not be defined in this content.The present invention also can be suitable for the camera of non-replacing camera lens formula.Pupil location and f-number obtain getting final product with suitable known method.

In addition, in the above-described embodiment, the example of handling by Electrofax 1 shot image data has been described, but may not be defined in this content.The present invention is to also being suitable for by the video camera shot image data of handling animation.In addition, for also being suitable for by the shot image data such as portable phone that have camera.That is, thus for be suitable for that imaging apparatus takes view data, can use the present invention.

In addition, in the above-described embodiment, explained in PC (PC) 31 and to have handled by electronic camera 1 shot image data and remove the example of effect of dust, but may not be defined in this content.Also this program can be installed on electronic camera 1.In addition, also this program can be installed on printer or projection arrangement etc.That is, to all devices of image data processing, the present invention can be suitable for.

The program of in PC 31, carrying out can provide through the data-signal of recording medium such as CD-ROM or internet etc.Figure 18 is the figure of its appearance of expression.PC 31 receives providing of program through CD-ROM 34.In addition, PC 31 has the function that is connected with communication line 401.Computer 402 provides the server computer of said procedure, stored program in hard disk 403 recording mediums such as grade.Communication line 401 is communication line or dedicated communication lines of internet, PC communication etc. etc.Computer 402 uses hard disk 403 read routines, and via communication line 401 program is sent among the PC 31.That is, with program as data signals carry in carrying ripple, and send via communication line 401.So, but program can be used as recording medium or the computer program product of carrying the computer of various ways such as ripple to read in provides.

More than, all execution modes and variation have been described, but the present invention is not limited to these contents.Other modes in the technological thought scope of the present invention are also contained in the scope of the present invention.

Claims (11)

1. image processing apparatus has:

Image acquiring unit obtains first image of taking through optical system and second image of taking with the optical condition different with said first image; With

Defect information preparing department utilizes said first image and said second image, makes the defect information that is comprised in said first image or said second image,

Said defect information preparing department in the said image that obtains, according to the mean value of the value of a plurality of pixels in the value of object pixel and the preset range that comprises this object pixel, makes the defect information in the said image,

Between said first image and said second image, at least one optical condition of f-number and pupil location is different.

2. image processing apparatus according to claim 1,

Have correction portion, utilize said defect information, proofread and correct the defective in said first image or said second image.

3. image processing apparatus according to claim 1,

Said defect information preparing department has the optical condition transformation component, at least one image of said first image and said second image is carried out conversion, so that said first image and said second image become the optical condition uniform images.

4. image processing apparatus according to claim 3,

Said optical condition transformation component at the f-number of said optical condition not simultaneously, carries out low pass filter to the picture element signal based on said first image or said second image and handles, and is transformed to the state of the defective of supposition when identical f-number.

5. image processing apparatus according to claim 4,

Said optical condition transformation component, utilizing in fact all, the low pass filter of equal weight carries out conversion.

6. image processing apparatus according to claim 3,

Said optical condition transformation component; In the pupil location of said optical condition not simultaneously; To picture element signal based on said first image or said second image; Carry out being transformed to the state that supposition is positioned at the defective of identical pupil location from the optical axis center of said optical system shifting processing to the skew of radius vector direction.

7. image processing apparatus according to claim 6,

Said optical condition transformation component, carry out along with from optical axis center away from and shifting processing that the amount that squints in the radius vector direction increases.

8. image processing apparatus according to claim 6,

Said optical condition transformation component supposes that the foreign matter of the reason that becomes said defective is positioned at the position that preset distance is arranged along the shooting mask of optical axis and said optical system, predicts the computing of shift amount, carries out shifting processing.

9. image processing apparatus according to claim 1,

Any one of said first image and second image is that another is the benchmark image that is used to make defect information as the calibration object image of the object of proofreading and correct.

10. image processing apparatus according to claim 9,

Said image acquiring unit obtains the said benchmark image of taking with the f-number that in the iris ring value of said optical system, turns to minimum state.

11. image processing apparatus according to claim 9,

Said image acquiring unit obtains the benchmark image of taking in the scheduled time before and after the shooting of said calibration object image.

Images