CN107979715A - Camera device - Google Patents
Camera device Download PDFInfo
- Publication number
- CN107979715A CN107979715A CN201710318575.1A CN201710318575A CN107979715A CN 107979715 A CN107979715 A CN 107979715A CN 201710318575 A CN201710318575 A CN 201710318575A CN 107979715 A CN107979715 A CN 107979715A
- Authority
- CN
- China
- Prior art keywords
- image
- camera device
- correction data
- correction
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The present invention provides a kind of camera device, it efficiently can correct image quality deterioration caused by image quality deterioration caused by the optical characteristics of the lens in camera device and image magnification processing when zooming, and natural high-definition picture can be also obtained to the image for carrying out digital zoom.Camera device has optical lens (1) and solid-state imager (2).Camera device has storage unit, its store correction data, the correction data be based on as solid-state imager (2) export and be divided into the point spread function obtained in the regional of the image of multiple regions according to optical lens (1) and image by digital zoom and image magnification when function out of focus both calculate.Camera device has image flame detection operational part (7), it is directed to each taken image, from correction data retention portion (6) extraction correction data, use the correction data extracted, for each image photographed, implement to be used for the calculation process for correcting the image deterioration based on point spread function and function out of focus.
Description
Technical field
The present invention relates to the camera device with optical lens and solid-state imager.
Background technology
Known digital camera usually has following camera devices, i.e. there is the camera device optical lens and solid to take the photograph
Element, view data is transformed to by the picture of the subject photographed, is then electronic signal by the image data transformation.Should
Camera device is applied not only to digital camera, is also assembled in mobile equipment i.e. smart mobile phone or tablet computer as camera module
In.The view data photographed by these camera devices, cause mainly due to optical aberration the light from point light source without
Method is converged on 1 point, and generation optics is out of focus and image quality deteriorates.Represent for point to expand with the function of the corresponding light diffusion out of focus
Dissipate function (PSF:Point Spread Function).The PSF is with the regional location (distance with picture centre of image：Picture
It is high) and change.
For above-mentioned due to processing out of focus caused by optical aberration, such as carrying out as shown below.That is, usually to due to saturating
The optical characteristics such as the optical aberration of mirror and the image that deteriorates carries out edge sharpening filtering process (Laplce's filtering etc.).But
If edge sharpening filtering process is excessive, easily unwanted overshoot or undershoot are produced in perimeter.In order to including image
Periphery inside obtain the image of high-quality, it is necessary to be corrected by PSF.Such as propose following technologies, i.e. for every
A camera module measures PSF, they are carried out calculation process as correction with data, regional to image and
Solid difference of each camera module etc. is corrected (referring for example to patent document 1).
Patent document 1：Japanese Unexamined Patent Publication 2010-177918 publications
On the other hand, in recent years, camera module is increasingly paid attention to used by being particularly the mobile equipment such as smart mobile phone
Zoom function.Optical zoom mechanic is needed in lens unit there are the moving mechanism of lens, and module size becomes larger, price becomes
Height, and can be intolerant to fall impacting.From this angle, mobile equipment is to the image magnification work(that is carried out by image procossing
Energy, i.e. so-called digital zoom functions become to pay attention to.In common digital zoom, implement using closest interpolation method, linearly
The interpolation processing of interpolation method, cubic convolution interpolation method etc. implements the processing such as edge sharpening filtering process, but is filtered in edge sharpening
It is as noted above in ripple processing, it is possible to produce overshoot or undershoot in adjacent edges and form unnatural image.In addition, line
Property interpolation or cubic convolution interpolation can not recover the radio-frequency component lost in sampling substantially.Thus, it is desirable to interpolation processing it
The outer processing for also carrying out recovering radio-frequency component.Thus, it is necessary to be answered respectively in the camera device commonly using digital zoom
To the processing due to image deterioration caused by optical characteristics and for the processing due to image deterioration caused by enhanced processing.
Therefore, the treating capacity increase of processing is recovered.In addition, magnifying power is changed using digital zoom in the film shooting of high frame per second
In the case of, it is desirable to the arithmetic processing apparatus of camera device has very high throughput, or the amplification to frame per second or digital zoom
The generations such as the change operation of rate limit.
The content of the invention
The present invention is exactly to propose in view of the foregoing, and its purpose is to provide a kind of camera device, it can be efficient
Ground caused by image quality deterioration caused by the optical characteristics of the lens in camera device and image magnification processing when zooming to drawing
Shoddyization is corrected, and makes the image of carry out digital zoom can also obtain natural high-definition picture.
In order to realize the purpose, camera device of the invention is characterized in that, with optical lens and solid-state image pickup
In the camera device of element, have：Storage unit, it stores correction data, which be based on by the solid-state image pickup
Element export and be divided into obtained in the regional of the image of multiple regions according to the optical lens point spread function,
And described image by digital zoom and image magnification when function out of focus both calculate；And calculation process list
Member, it is directed to each taken described image, extracts the correction data from the storage unit, uses what is extracted
The correction data, for each described image photographed, implement to be used for correct based on the point spread function and described
The calculation process of the deterioration of the described image of function out of focus.
According to said structure, in the camera device for carrying out digital zoom, for the optical characteristics (light due to optical lens
Learn aberration etc.) caused by the deterioration of image that photographs and based on caused by the image magnification processing carried out by digital zoom
Both deteriorations of the deterioration of the image photographed, can use integrated correction data efficiently to be corrected.
That is, can use point spread function relevant with the deterioration of image caused by the optical characteristics due to optical lens,
And due to the corresponding correction data of both relevant functions out of focus of deterioration of image caused by enhanced processing, efficiently rectify
Positive digital becomes defocused image.
In the structure of the present invention, the correction data are preferably stored in the storage list by following manner
In member, i.e. the point spread function is converted into after frequency domain by Fourier transformation respectively with the function out of focus and is multiplied each other,
The inverse of multiplication result of calculation is obtained, to the progress inverse Fourier transform reciprocal obtained, and passes through the warp in real space region
After product filtering is converted, it is stored in the storage unit.
According to said structure, it can use and spread corresponding put of image deterioration caused by the optical characteristics with optical lens
The correction data obtained after function and combination of function out of focus corresponding with image deterioration caused by enhanced processing, are efficiently corrected
Image after digital zoom.
In addition, in the structure of the present invention, preferably with edge detection unit, it is in the described image that photographs
Edge strength be detected,
The edge of the operation processing unit and each position of the described image detected by the edge detection unit is strong
Degree accordingly, adjusts the intensity of the correction carried out by the calculation process.
According to said structure, it is above-mentioned corrected based on point spread function and function out of focus when, can prevent opposite side
The less, light intensity that changes that the relatively low part of edge intensity, i.e. light intensity are produced as position is different is on the image of uniform state
Region carry out the correction identical with the part that edge strength is higher, this case that instead result in noise increase.Correct intensity
Difference can also be for example edge strength be less than setting in the case of without correction, in the case of higher than setting into
Row correction.Alternatively, it is also possible to set upper limit value and lower limit to edge strength, in the case where edge strength is in lower limit not
Corrected, more than for upper limit value in the case of carry out above-mentioned correction, between upper limit value and lower limit in the case of reduce
The intensity of above-mentioned correction and corrected.That is, edge strength can also be divided into several grades, with the degree with edge strength
Set appropriate correction intensity.
In addition, in the structure of the present invention, preferably also have：Multiple shooting units, it is different with focal length
The lens and the solid-state imager；And switch unit, it switches and comes from according to zoom ratio when shooting
The output of the described image of multiple shooting units,
The storage unit stores the correction data corresponding with each shooting unit,
The operation processing unit carries out following calculation process, i.e. the use correction number corresponding with each shooting unit
According to being corrected to the described image exported from the shooting unit.
According to said structure, by will export with the different multiple shooting units of focal length and from these shooting units
Image switch over, so as to change magnifying power in a manner of replacing commutative lens, in this structure, by will be right
The switching of image from multiple shooting units and digital zoom are combined, so that in performance close to the situation of optical zoom
Under, it can also be corrected by using above-mentioned correction data corresponding with each shooting unit, efficiently correcting image.
The effect of invention
In accordance with the invention it is possible to the point spread function of one of optical characteristics using lens used in camera device is efficient
Ground correcting image regional resolution ratio deterioration and by zoom operation carry out image magnification processing it is out of focus and what is produced,
The image of high image quality can more easily be obtained.
Brief description of the drawings
Fig. 1 is the block diagram for the camera device for representing the 1st embodiment of the present invention.
Fig. 2 is the block diagram of the image flame detection operational part for the camera device for representing the 1st embodiment.
Fig. 3 A are the figures of the local ideal image (A) for the resolution test plate for representing the 1st embodiment.
Fig. 3 B be represent the 1st embodiment the part to resolution test plate shoot obtained from deteriorate
The figure of image (B).
Fig. 3 C are the curve maps being compared to the luminance variations of image (A) and image (B) for representing the 1st embodiment.
Fig. 3 D be represent the 1st embodiment the luminance variations that image (B) occurs relative to image (A) carry out differential after
Value figure.
Fig. 4 A are the figures of the local ideal image (A) for the resolution test plate for representing the 1st embodiment.
Fig. 4 B are to represent that the part by resolution test plate of the 1st embodiment carries out shooting amplified deteriorate
Image (B) figure.
Fig. 4 C are the curve maps being compared to the luminance variations of image (A) and image (B) for representing the 1st embodiment.
Fig. 4 D be represent the 1st embodiment the luminance variations that image (B) occurs relative to image (A) carry out differential after
Value figure.
Fig. 5 is the block diagram for the camera device for representing the 2nd embodiment of the present invention.
Fig. 6 A are the figures of image for representing to photograph using a lens.
Fig. 6 B are the figures of the restored image for the image for representing Fig. 6 A.
Fig. 6 C are the figures of image for representing to photograph using another lens.
Fig. 6 D are the figures of the restored image for the image for representing Fig. 6 C.
Embodiment
Hereinafter, embodiments of the present invention are illustrated.
(the 1st embodiment)
First, the 1st embodiment of the present invention is illustrated.
As shown in Figure 1, the camera device involved by the 1st embodiment of the present invention has：Optical lens 1, it is used to make
Photoimaging from subject；Solid-state imager 2, the picture of subject is being imaged by it by optical lens 1 in photosurface
When, light is transformed to signal Electricity lotuses, view data is generated and exports the picture signal for representing the view data；Demosaicking portion
3, it carries out demosaicking processing (interpolation processing) to the picture signal exported from solid-state imager 2；Enlargement processing section 4, its
View data after demosaicking is handled is amplified by zoom；YC separation units 5, it will represent the rgb signal of view data
It is transformed to luminance signal and colour difference signal；And storage circuit (memory) corrects data retention portion 6, it keeps being used to recover
The correction data of view data.
In addition, camera device has：Image flame detection operational part 7, it uses the correction being stored in correction data retention portion 6
Data, carry out the computing for being corrected to above-mentioned luminance signal；Timing adjustment portion 8, its make the timing of colour difference signal with after correction
Luminance signal coordinate；Noise reduction portion 9, it reduces unwanted caused by processing in image flame detection operational part 7 make an uproar
Sound；And combining unit 10, the luminance signal after correction and the colour difference signal after delay are synthesized and exported by it.
Above-mentioned image flame detection operational part 7 has：As the view data maintaining part 11 of storage part, it is used to keep image to rectify
The view data of N × N needed for positive computing；De-convolution operation portion 12, it carries out deconvolution filtering processing；Edge detection portion 13,
The edge strength of its detection image data；Luminance difference calculating part 14, it calculates the luminance difference in the view data of N × N；Addition is transported
Calculation portion 15, it calculates pixel value according to the output valve in de-convolution operation portion 12 and edge detection portion 13；And color matrix adjustment
Portion 16, it carries out color adjustment.
Optical lens 1 can be lens group that is a piece of or being made of multi-disc.Solid-state imager 2 for example can be with
It is the element using CCD or CMOS etc..In addition, solid-state imager 2 is provided with chromatic filter, generation and chromatic filter
The corresponding signal Electricity lotuses of each color, the value of signal Electricity lotuses is circuitry digitizing by built-in analog-to-digital conversion, from
And obtain view data.
Demosaicking portion 3 carries out demosaicking processing, the picture with surrounding to the view data exported from solid-state imager 2
Prime number is according to accordingly calculating (interpolation：Interpolation) pixel value of color outside each pixel color of chromatic filter is obtained, such as
Chromatic filter is calculated as the blueness in red pixel, the value of green, is calculated in pixel of the chromatic filter for green
Blueness, red value, calculate chromatic filter as red, the value of green in the pixel of blueness.
The enlargement processing section 4 and zoom instructions a based on user's operation etc. accordingly, will be carried out by demosaicking portion 3
View data after demosaicking processing is amplified with any multiplying power.At this time, pixel number is increased using interpolation.In addition, do not becoming
In the case that Jiao's instruction a is i.e. without instruction amplification, enlargement processing section 4 for example directly exports the image inputted.
The RGB data as view data that is exported from enlargement processing section 4 is transformed to briliancy, aberration by YC separation units 5
Data, using human eye to the sensitive this case of briliancy, only carry out correction process described later to briliancy (Y).
Correction data retention portion 6 is for keeping depositing for the correction data of being recovered obtained view data
Storing up electricity road (memory), based on the position signal b of the position on zoom instructions a and expression view data, switches and is kept strong
Correction data simultaneously exports.Herein, magnifying power when view data to be carried out to zoom is included in zoom instructions a.In addition, position signal
B is the signal that the position in view data is represented such as with coordinate, shows the position in the view data before zoom.Becoming
Jiao Shi, can be using as the position of basic point in the case where the amplification basic point of original image necessarily be in the position identical with original image
Install as coordinate origin, position can also be represented with the distance with basic point.
It is as shown in the above to correct data, by for the bad of view data caused by the optical characteristics such as the optical aberration to lens
Change the optical correction data recovered and for the bad of the view data caused by the enhanced processing as digital zoom
Change recovered multiplying power correction data combination obtained from, by the position on original digital image data with due to magnifying power difference without
Same correction data are stored in correction data retention portion 6.In addition, in the case of magnifying power difference, meter may be designed in
Parameter corresponding with magnifying power is calculated, i.e., calculates rectify corresponding with magnifying power by computing from the correction data as benchmark
Correction data.
While the luminance signal of the view data from YC separation units 5 is read, reading comes from image flame detection operational part 7
The correction data of data retention portion 6 are corrected, the correction of view data is carried out based on the content.For image flame detection operational part 7
Detailed content is described below.
Timing adjustment portion 8 coordinates for the timing for the colour difference signal for making to export from YC separation units 5 with image flame detection operational part 7
And carry out delay disposal.
Noise reduction portion 9 is used to reduce unwanted noise caused by processing in image flame detection operational part 7, can be with
Suitably using medium filtering, Yi Pu sirons filtering (Epsilon filter), bilateral filtering etc., neighboring pixel can also be judged
Luminance information (absolute value of briliancy, luminance difference etc.) and suitably change filtering strength.Above-mentioned noise reduces processing according to feelings
Condition can also be omitted.
Combining unit 10 is by from the luminance signal after the recovery that noise reduction portion 9 exports and the color exported from timing adjustment portion 8
Composite Signal, is as needed RGB, YUV, RAW etc. by image format conversion, as the last output from image device
Output image data.
As shown in Fig. 2, the view data maintaining part 11 in image flame detection operational part 7 is used to store from YC separation units 5
The luminance signal of view data, the picture number of N × N pixels needed for image flame detection computing is stored for each pixel respectively
According to.
De-convolution operation portion 12 for each key element respectively by from view data maintaining part 11 input come N × N briliancy
Signal is multiplied with the correction data read from correction data retention portion 6, and then carries out add operation and obtain the figure of above-mentioned N × N
As pixel value (the output valve d) in de-convolution operation portion 12 positioned at center in data.In the present embodiment, it is set to N
=9.
Edge detection portion 13 calculates the vertical of image using central part M × M in the data of view data maintaining part 11
And the edge strength in horizontal direction, and the output valve e using them as edge detection portion 13.M=3 is used in edge detection
Sobel filtering calculate it is square in vertical and horizontal directions.In addition, edge strength can also be obtained using other algorithms.
According to the output valve in de-convolution operation portion 12, edge detection portion 13, using addition operation division 15 in terms of following algorithm
Calculate the pixel value of object pixel.
【Formula 1】
Herein：
d：The output valve (pixel value after correction) in de-convolution operation portion 12
e：The output valve in edge detection portion 13
I：Target pixel value (pixel value before correction)
th_H、th_L：Edge strength threshold value
α：The mixed proportion of de-convolution operation result and original image
The parameter of described th_H, th_L, α etc. can determine suitably, can be in design or when making sample or manufacture
The observation image such as during finished product and suitably determine, can also be set as suitably changing based on photographed scene sometimes.
Processing in the addition operation division 15 is that the correction process of image is being carried out by de-convolution operation portion 12 described later
Carry out afterwards, correction intensity is changed according to the output valve e in edge detection portion 13.The output valve e in edge detection portion 13 is for example above-mentioned
It is shown to set two threshold values th_H, th_L up and down.In the situation for below the threshold value th_L that the output valve e of edge strength is downside
Under, by the output valve from addition operation division 15 be set to correction before view data object pixel pixel value I.That is, for
The output valve e of edge strength in each pixel of view data is the pixel of below downside threshold value th_L, not correcting image data
Pixel value.
In the case where the output valve e of edge strength is more than the threshold value th_H of upside, by from addition operation division 15
Output valve is set to the pixel value of the object pixel of the view data after correction, i.e. the output valve d in de-convolution operation portion 12.That is, it is right
The output valve e of edge strength in each pixel of view data is the pixel of more than upside threshold value th_H, and pixel value is carried out
Correction.
, will in the case where the output valve e of edge strength falls between the threshold value th_H of upside and the threshold value th_L of downside
Output valve d from addition operation division 15 is set to after the pixel value I of the object pixel of the view data before correction and correction
The value that the output valve d in de-convolution operation portion 12 is synthesized into.That is, for the edge strength in each pixel of view data
Output valve e falls the pixel between downside threshold value th_L and upside threshold value th_H, and pixel value is set to the pixel value I before correction
Value between the pixel value after correction.
In addition it is also possible to the threshold value of edge strength is set to 1, more than threshold value in the case of, add operation will be come from
The output valve in portion 15 is set to the pixel value of the object pixel of the view data after correction, i.e. the output valve d in de-convolution operation portion 12,
In the case where the output valve e of edge strength is below threshold value, before the output valve from addition operation division 15 is set to correction
The pixel value I of the object pixel of view data.Alternatively, it is also possible to set the threshold value of more than 3, pixel value before correction and rectify
The picture for being synthesized into the pixel value before correction and the pixel value after correction in embedded multiple stages between pixel value after just
Element value.In this way, in the relatively low part of edge strength, by that without correcting or reducing correction intensity, can prevent to position
Part different and that luminance variations are less carries out correction and instead results in luminance variations becoming larger, so as to produce the situation of noise.
Output from addition operation division 15 (image flame detection operational part 7) passes through with the colour difference signal from YC separation units 5
Combining unit 10 synthesizes, and reverts to rgb signal.Then carry out color adjustment using color matrix adjustment part 16 and obtain final figure
As data.
In the following, using schematic diagram, illustrate to utilize letter out of focus out of focus caused by the point spread function (PSF) of lens and amplification
Number generation image flame detection data.
Fig. 3 A are using as the subject after the partial enlargement of the resolution test plate of subject, are pair in other words
Subject shot after the ideal image (A) without deterioration.Fig. 3 B be a same part is shot by camera device and
Any processing, image (B) there are deterioration are not carried out.Fig. 3 C be with as Fig. 3 A, Fig. 3 B arrow shown in image (A),
(B) the corresponding figure of situation being scanned, is when being based on each pixel and one-dimensionally representing to shoot original subject
Ideal image (A) in the change that occurs with the position on arrow of light intensity (briliancy) and amplified image (B) in
The figure of change that occurs with the position on arrow of light intensity (briliancy).In addition, transverse axis is represented at arrow in units of pixel
Each pixel, the longitudinal axis represent briliancy (intensity), and intensity is standardized in a manner of as same ratio.Fig. 3 D are the figures to Fig. 3 B
Picture (B) carries out the figure of differential relative to the luminance variations of the image (A) of Fig. 3 A, equivalent to the point spread function of lens, in solid
Diffusion profile on the photosurface of photographing element 2.That is, implemented equivalent to by the luminance variations of Fig. 3 (A) in real space with Fig. 3 D
Convolution integral (convolution) as a result, it becomes the image (B) of the i.e. Fig. 3 B of image that actually photographed.
In order to by obtained image (B) revert to ideal image (A), it is known that by by shown in Fig. 3 D and Fig. 3 C
Image B luminance variations carry out Fourier transformation and be unfolded in spatial frequency domain, divided by after the value shown in Fig. 3 D carry out Fu in
Leaf inverse transformation.
On the other hand, by zoom handle carry out image magnification when, for example, if keep original pixels quantity it is constant and
The area expansion of enlarged drawing size, then single pixel.In the case, due to image augmentation but pixel quantity is constant, so
It is observed that the resolution ratio of image reduces.Therefore, the image that amplifies in the case where increasing pixel quantity using interpolation, with it is straight
Connect and original image is enlarged into the image of same size compares, compared with the Pixel Dimensions of original image, utilize the figure of interpolation amplification
The pixel of picture is smaller.
As described above, it is necessary to by making the Pixel Dimensions of enlarged drawing diminish relative to the Pixel Dimensions of original image, so that
Spatial frequency is improved as shown in Figure 4, but if being usually used simple interpolation processing, original sampling frequency reduces, so
It is able to observe that resolution ratio deteriorates.
Herein, Fig. 4 A and Fig. 4 B show the local image of resolution test plate, and Fig. 4 A show preferable before amplifying
Original image, that is, image (A), Fig. 4 B show amplified image i.e. image (B).Fig. 4 (C) show as Fig. 4 A, Fig. 4 B arrow shown in
The change of briliancy (light intensity) in the case of being scanned to image (A), (B), transverse axis represent image (A), (B) in arrow
The pixel of each position at place, the longitudinal axis represent briliancy.
The deterioration of resolution ratio is due to caused by luminance variations become gentle as shown in Figure 4 C, passes through the image to Fig. 4 A
(A) function out of focus shown in trellis diagram 4D and obtain the image (B) of amplified Fig. 4 B so that resolution ratio deteriorate.Thus, with it is preceding
In the same manner, the briliancy of the image (B) of Fig. 4 B is become for the correction that the resolution ratio that the point spread function using lens stated carries out deteriorates
Change and the value of the function out of focus of Fig. 4 D carry out Fourier transformation respectively, and divided by Fig. 4 D function out of focus value after, by Fu
Leaf inverse transformation is restored, so as to obtain the enlarged drawing roughly the same with the resolution ratio of image (A).
Expression is proceeded as described below as formula.
G：The briliancy distribution of subject, I：The briliancy distribution of the image photographed, P：Point spread function
I=G*P ... (1)
F (I)=F (G*P)=F (G) × F (P) ... (2)
G=F-1(F (I)/F (P))=I*F-1(1/F(P))……(3)
In addition, symbol * represents convolution integral, F (), F-1() represents Fourier transformation and inverse Fourier transform respectively.Usually
F-1 (1/F (P)) is referred to as deconvolution filtering.
In addition, statement is proceeded as described below in image magnification.
I '=I*B=G*P*B ... (4)
Here, I '：Enlarged drawing, B：Amplify the function out of focus produced
It is such as following shown in order to revert to G from I '.
G=I*F-1(1/F(P))*F-1(1/F(B))……(5)
Since deconvolution filtering is usually showed with the matrix of (2N+1) × (2N+1), so the calculation process of 1 pixel needs
Want the secondary multiplying processing of (2N+1) × (2N+1) and add operation processing.
Need gradually to carry out calculation process above-mentioned twice in image magnification, in addition 1 rank convolution integral in order to obtain
As a result, the view data of next group of N row to be entered such as at least also need to.In the present invention, by the way that the formula of above-mentioned (5) is as follows
It is shown to be deformed, precalculate correction data (deconvolution filtering) corresponding with multiplying power and keep, so as to reduce computing
Number.
=I*F-1(1/ (F (P) × 1/F (B)))=I*F-1(1/(F(P)×F(B)))……(6)
In addition, the point spread function of lens can according to circumstances use the reality after optical design value, camera device assembling
Measured value or it is replaced into appropriate function based on above-mentioned value etc..Gaussian function, long-range navigation can be enumerated as appropriate function
Hereby function, appropriate multinomial etc..In the case where using optical design value, measured value, image quality can be observed while fitting
Correct enlargement range and select optimal value in locality.
(the 2nd embodiment)
In the following, the 2nd embodiment of the explanation present invention.
As shown in figure 5, the camera device involved by the 2nd embodiment of the present invention is in the 1st embodiment shown in Fig. 1
In camera device, in addition to optical lens 1 and solid-state imager 2, optical lens 1 ' and solid-state imager also add
2 ', and added switching circuit 17, it switches output signal from solid-state imager 2 and from solid-state imager
2 ' output signal.Optical lens 1 and solid-state imager 2, optical lens 1 ' and solid-state imager 2 ' respectively constitute shooting
Unit.
Optical lens 1 and optical lens 1 ' are different using the different lens of field angle or focal length, optical power.Such as
If the ratio between focal length of optical lens 1 ' and optical lens 1 is 1：2, then equivalent to 2 times of zooms optically, to take the photograph
Make to have the solid-state imager 2 of optical lens 1 to export signal in the case of being 1≤Z ＞ 2 as the zoom ratio Z of device entirety,
Make to have the solid-state imager 2 ' of optical lens 1 ' to export the mode of signal in the case of Z≤2, by switching circuit 17 into
Row switching.Other structures are identical with the camera device of the 1st embodiment.
In addition, solid-state imager 2 and 2 ' can also be with different size, but in the feelings of the latter with same size
It is adjusted under condition, it is necessary to carry out zoom ratio adjustment etc. according to different size.
In the 2nd embodiment, the view data taken by each shooting unit is also carried out in the 1st embodiment into
Capable above-mentioned processing.Thus, in the camera device of the 2nd embodiment, for example, by switch output image shooting unit or
The signal for the image that person's switching exports respectively from the two shooting units, so as to the corresponding level of the quantity of shooting unit
Number and hierarchically by image magnification ratio optically make a change, in addition when continuously changing magnifying power using digital zoom, as above
Shown in stating, correction data are generated based on the function out of focus that the point spread function according to optical characteristics and enhanced processing produce and are deposited
Storage, thus, it is possible to efficiently be carried out using the correction data to the deterioration due to image caused by optical characteristics and enhanced processing
Recover.
Fig. 6 shows the part for the image that camera device according to the present invention obtains.
Fig. 6 A show the normal image obtained by optical lens 1 and the combination of solid-state imager 2, and Fig. 6 B are by the 1st
The antidote of embodiment normal image is corrected after correcting image, Fig. 6 C, Fig. 6 D be zoom ratio is set to 2 feelings
The image that the side of optical lens 1 ' under condition obtains, similarly shows normal image and correcting image.
Any situation can be confirmed has obtained good image quality using the present invention.
In addition, in the camera device of the respective embodiments described above, to the output signal from solid-state imager 2 (2 ')
The calculation process part handled can be located at as in the camera module of camera device, can also be located at camera module
Outside, in this case, it is possible to which calculation process part by camera module and outside forms camera device.In addition,
Calculation process part can use special circuit, can also use the universal circuit after programming, can also use special circuit
With the circuit of universal circuit combination.In addition, being assembled in smart mobile phone or tablet computer etc. in camera device has general computing
In the case of in the electronic equipment of processing unit, can also by above-mentioned calculation process part all or part of be located at based on electricity
The application (program) of sub- equipment side and in the arithmetic processing apparatus of electronic equipment side that acts.
The explanation of label
1 optical lens
2 solid-state imagers
6 correction data retention portions (storage unit)
7 image flame detection operational parts (operation processing unit)
13 edge detection portions (edge detection unit)
17 switching circuits (switch unit).
Claims (4)
1. a kind of camera device, it is with optical lens and solid-state imager, it is characterised in that
The camera device has：Storage unit, it stores correction data, which be based on by solid-state image pickup member
Part export and be divided into obtained in the regional of the image of multiple regions according to the optical lens point spread function, with
And described image by digital zoom and image magnification when function out of focus both calculate；And
Operation processing unit, it is directed to each taken described image, and the correction number is extracted from the storage unit
According to using the correction data extracted, for each described image photographed, implementation, which is used to correct, is based on the point
The calculation process of the deterioration of the described image of spread function and the function out of focus.
2. camera device according to claim 1, it is characterised in that
The correction data are stored in the storage unit by following manner, i.e. by the point spread function and the mistake
Burnt function is converted into after frequency domain by Fourier transformation respectively to be multiplied each other, the inverse of multiplication result of calculation is obtained, to what is obtained
Inverse carries out inverse Fourier transform, and after being converted by the deconvolution filtering in real space region, is stored in the storage
In unit.
3. camera device according to claim 1 or 2, it is characterised in that
With edge detection unit, it is detected the edge strength in the described image that photographs,
The edge strength pair of each position of the operation processing unit and the described image detected by the edge detection unit
Ying Di, adjusts the intensity of the correction carried out by the calculation process.
4. camera device according to any one of claim 1 to 3, it is characterised in that
Have：Multiple shooting units, it is with the different lens of focal length and the solid-state imager；And cut
Unit is changed, it switches the output of the described image from multiple shooting units according to zoom ratio when shooting,
The storage unit stores the correction data corresponding with each shooting unit,
The operation processing unit carries out following calculation process, i.e. and the use correction data corresponding with each shooting unit are right
Corrected from the described image of shooting unit output.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2016206729A JP6857006B2 (en) | 2016-10-21 | 2016-10-21 | Imaging device |
| JPJP2016-206729 | 2016-10-21 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107979715A true CN107979715A (en) | 2018-05-01 |
| CN107979715B CN107979715B (en) | 2021-04-13 |
Family
ID=62012188
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710318575.1A Expired - Fee Related CN107979715B (en) | 2016-10-21 | 2017-05-08 | Image pickup apparatus |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP6857006B2 (en) |
| CN (1) | CN107979715B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020107995A1 (en) * | 2018-11-26 | 2020-06-04 | Oppo广东移动通信有限公司 | Imaging method and apparatus, electronic device, and computer readable storage medium |
| CN114630008A (en) * | 2020-12-10 | 2022-06-14 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004328506A (en) * | 2003-04-25 | 2004-11-18 | Sony Corp | Imaging apparatus and image recovery method |
| CN101222583A (en) * | 2007-01-09 | 2008-07-16 | 奥林巴斯映像株式会社 | Imaging apparatus adapted to implement electrical image restoration processing |
| CN102930507A (en) * | 2011-08-08 | 2013-02-13 | 佳能株式会社 | Image processing method, image processing apparatus, and image pickup apparatus |
| CN103201765A (en) * | 2010-09-28 | 2013-07-10 | 马普科技促进协会 | Method and device for recovering a digital image from a sequence of observed digital images |
| CN105684417A (en) * | 2013-10-31 | 2016-06-15 | 富士胶片株式会社 | Image processing device, image capture device, parameter generating method, image processing method, and program |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH06261238A (en) * | 1993-03-05 | 1994-09-16 | Canon Inc | Image pickup device |
| JP4341019B2 (en) * | 2004-01-09 | 2009-10-07 | カシオ計算機株式会社 | Imaging apparatus and program |
| JP2010141661A (en) * | 2008-12-12 | 2010-06-24 | Toshiba Corp | Image processing device |
| US8553106B2 (en) * | 2009-05-04 | 2013-10-08 | Digitaloptics Corporation | Dual lens digital zoom |
| JP2013021509A (en) * | 2011-07-11 | 2013-01-31 | Canon Inc | Image processing apparatus and method |
-
2016
- 2016-10-21 JP JP2016206729A patent/JP6857006B2/en active Active
-
2017
- 2017-05-08 CN CN201710318575.1A patent/CN107979715B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004328506A (en) * | 2003-04-25 | 2004-11-18 | Sony Corp | Imaging apparatus and image recovery method |
| CN101222583A (en) * | 2007-01-09 | 2008-07-16 | 奥林巴斯映像株式会社 | Imaging apparatus adapted to implement electrical image restoration processing |
| CN103201765A (en) * | 2010-09-28 | 2013-07-10 | 马普科技促进协会 | Method and device for recovering a digital image from a sequence of observed digital images |
| CN102930507A (en) * | 2011-08-08 | 2013-02-13 | 佳能株式会社 | Image processing method, image processing apparatus, and image pickup apparatus |
| CN105684417A (en) * | 2013-10-31 | 2016-06-15 | 富士胶片株式会社 | Image processing device, image capture device, parameter generating method, image processing method, and program |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020107995A1 (en) * | 2018-11-26 | 2020-06-04 | Oppo广东移动通信有限公司 | Imaging method and apparatus, electronic device, and computer readable storage medium |
| CN114630008A (en) * | 2020-12-10 | 2022-06-14 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
| CN114630008B (en) * | 2020-12-10 | 2023-09-12 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107979715B (en) | 2021-04-13 |
| JP6857006B2 (en) | 2021-04-14 |
| JP2018067868A (en) | 2018-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6347675B2 (en) | Image processing apparatus, imaging apparatus, image processing method, imaging method, and program | |
| US8350948B2 (en) | Image device which bypasses blurring restoration during a through image | |
| US9581436B2 (en) | Image processing device, image capturing apparatus, and image processing method | |
| JP4818957B2 (en) | Imaging apparatus and method thereof | |
| US8340512B2 (en) | Auto focus technique in an image capture device | |
| Guichard et al. | Extended depth-of-field using sharpness transport across color channels | |
| TWI554106B (en) | Method and image capturing device for generating image bokeh effect | |
| US9369693B2 (en) | Stereoscopic imaging device and shading correction method | |
| KR20120068655A (en) | Method and camera device for capturing iris or subject of good quality with one bandpass filter passing both visible ray and near infra red ray | |
| JP2005286482A (en) | Distortion correcting apparatus and imaging apparatus provided with distortion correcting apparatus | |
| US20100245632A1 (en) | Noise reduction method for video signal and image pickup apparatus | |
| JP2009260620A (en) | Image processing apparatus and method | |
| CN107079127A (en) | Projecting apparatus and image display method | |
| US20140184859A1 (en) | Imaging device | |
| US10559068B2 (en) | Image processing device, image processing method, and program processing image which is developed as a panorama | |
| KR20070004202A (en) | Method for correcting lens distortion in digital camera | |
| JP5159715B2 (en) | Image processing device | |
| CN107979715A (en) | Camera device | |
| JP6976754B2 (en) | Image processing equipment and image processing methods, imaging equipment, programs | |
| Tisse et al. | Extended depth-of-field (EDoF) using sharpness transport across colour channels | |
| CN110784642B (en) | Image processing apparatus, control method thereof, storage medium, and imaging apparatus | |
| JP6961423B2 (en) | Image processing equipment, imaging equipment, control methods for image processing equipment, programs and recording media | |
| US8736722B2 (en) | Enhanced image capture sharpening | |
| JP4364847B2 (en) | Imaging apparatus and image conversion method | |
| JP5513310B2 (en) | Imaging module, image signal processing method, restoration processing apparatus and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210413 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |