CN105122793B - Image processing device, image capture device, and image processing program - Google Patents
Image processing device, image capture device, and image processing program Download PDFInfo
- Publication number
- CN105122793B CN105122793B CN201480018899.8A CN201480018899A CN105122793B CN 105122793 B CN105122793 B CN 105122793B CN 201480018899 A CN201480018899 A CN 201480018899A CN 105122793 B CN105122793 B CN 105122793B
- Authority
- CN
- China
- Prior art keywords
- data
- image
- parallax
- dimensional
- viewpoint direction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 57
- 238000009826 distribution Methods 0.000 claims abstract description 117
- 238000009499 grossing Methods 0.000 claims abstract description 102
- 238000012937 correction Methods 0.000 claims abstract description 16
- 238000003384 imaging method Methods 0.000 claims description 60
- 230000007246 mechanism Effects 0.000 claims description 50
- 230000003287 optical effect Effects 0.000 claims description 42
- 230000008859 change Effects 0.000 claims description 18
- 239000000203 mixture Substances 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 12
- 238000005096 rolling process Methods 0.000 claims description 5
- 238000003672 processing method Methods 0.000 claims 2
- 238000012423 maintenance Methods 0.000 claims 1
- 238000000034 method Methods 0.000 description 79
- 238000006243 chemical reaction Methods 0.000 description 38
- 230000008569 process Effects 0.000 description 35
- 210000001747 pupil Anatomy 0.000 description 34
- 230000006870 function Effects 0.000 description 24
- 238000009877 rendering Methods 0.000 description 12
- 230000000694 effects Effects 0.000 description 11
- 239000003086 colorant Substances 0.000 description 10
- 230000015654 memory Effects 0.000 description 10
- 238000003860 storage Methods 0.000 description 10
- 210000003128 head Anatomy 0.000 description 9
- 238000005286 illumination Methods 0.000 description 9
- 210000000162 simple eye Anatomy 0.000 description 9
- 238000005520 cutting process Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 238000011282 treatment Methods 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 6
- 230000015572 biosynthetic process Effects 0.000 description 6
- 239000000523 sample Substances 0.000 description 6
- 238000003786 synthesis reaction Methods 0.000 description 6
- 238000002156 mixing Methods 0.000 description 5
- 238000013459 approach Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 4
- 230000033228 biological regulation Effects 0.000 description 4
- 230000008030 elimination Effects 0.000 description 4
- 238000003379 elimination reaction Methods 0.000 description 4
- 239000004744 fabric Substances 0.000 description 4
- 230000010354 integration Effects 0.000 description 4
- 239000000047 product Substances 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 230000002207 retinal effect Effects 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000003321 amplification Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 210000000887 face Anatomy 0.000 description 2
- 230000008571 general function Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000009738 saturating Methods 0.000 description 2
- 241000894007 species Species 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 241001062009 Indigofera Species 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 229910052681 coesite Inorganic materials 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 229910052906 cristobalite Inorganic materials 0.000 description 1
- 230000009365 direct transmission Effects 0.000 description 1
- 238000005530 etching Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000116 mitigating effect Effects 0.000 description 1
- 238000002715 modification method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000000377 silicon dioxide Substances 0.000 description 1
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N silicon dioxide Inorganic materials O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 229910052682 stishovite Inorganic materials 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 229910052905 tridymite Inorganic materials 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
- H04N13/31—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays using parallax barriers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/296—Synchronisation thereof; Control thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/76—Circuitry for compensating brightness variation in the scene by influencing the image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
- H04N13/232—Image signal generators using stereoscopic image cameras using a single 2D image sensor using fly-eye lenses, e.g. arrangements of circular lenses
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
Provided is an image processing device, comprising: a means for receiving input of at least two disparity images, being an image of a first viewpoint direction and an image of a second viewpoint direction; a means for respectively generating, for the image of the first viewpoint direction and the image of the second viewpoint direction, a first smoothed image and a second smoothed image at least by applying a smoothing filter in a direction which joins the first and second viewpoints; a means for generating a reference image which is configured with a value based on the product of a value of the first smoothed image and a value of the second smoothed image; a means for generating gain distribution data by dividing the value of the reference image by the value of the first smoothed image; a means for carrying out a gain correction, with respect to the image of the first viewpoint direction, for each pixel on the basis of the gain distribution data; and a means for outputting the gain corrected image of the first viewpoint direction as a disparity image. The smoothing filter has a filter diameter of the same order as the degree of disparity which arises between the image of the first viewpoint direction and the image of the second viewpoint direction.
Description
Technical field
The present invention relates to image processing apparatus, camera head and image processing program.
Background technology
It is known to following camera head：Using single photographic optical system, have each other parallax by once shooting to generate
Left and right anaglyph.
Prior art literature
Patent document
Patent document 1：Japanese Unexamined Patent Publication 2003-7994 publications
The content of the invention
In the anaglyph of the left and right obtained by this camera head, shading value different from each other is generated
(shading)。
The image processing apparatus of the 1st mode of the present invention are characterised by having：Input is by an optical system to complete
The subject of the segment beam passed through from region different from each other in portion's light beam the 1st viewpoint side as obtained from being shot
To image and the 2nd viewpoint direction image at least two anaglyphs mechanism；Image and the 2nd to the 1st viewpoint direction
The image of viewpoint direction at least applies smoothing wave filter on the direction for connecting the 1st viewpoint and the 2nd viewpoint respectively, thus divides
Not Sheng Cheng the 1st smoothing image and the 2nd smoothing image mechanism；Generate and smoothed with the 2nd by the value that image is smoothed based on the 1st
Change the mechanism of the benchmark image of the value composition of the product of the value of image；By the way that the value of benchmark image is smoothed into image divided by the 1st
Value is generating the mechanism of the 1st gain profiles data；Each picture is carried out to the image of the 1st viewpoint direction based on the 1st gain profiles data
The mechanism of the gain-boosted op amp of element；And the image of the 1st viewpoint direction after gain-boosted op amp will be carried out as stereoscopic display
Anaglyph and the mechanism that exports, it is have the figure with the image in the 1st viewpoint direction and the 2nd viewpoint direction to smooth wave filter
The wave filter that the parallax amount produced as between is smoothed for the carrying out of the filtering diameter of same degree.
The image processing apparatus of the 2nd mode of the present invention are characterised by having：Input is by an optical system to complete
The subject of the segment beam passed through from region different from each other in portion's light beam the 1st viewpoint side as obtained from being shot
To image and the 2nd viewpoint direction image at least two anaglyphs mechanism；Image and the 2nd to the 1st viewpoint direction
The image of viewpoint direction at least applies smoothing wave filter on the direction for connecting the 1st viewpoint and the 2nd viewpoint respectively, thus divides
Not Sheng Cheng the 1st smoothing image and the 2nd smoothing image mechanism；Based on the 1st smoothing image and the 2nd smoothing image next life
Into the mechanism of benchmark image；By the way that the value of benchmark image is generated into the 1st gain profiles data divided by the value of the 1st smoothing image
Mechanism；By the way that the value of benchmark image is generated into the mechanism of the 2nd gain profiles data divided by the value of the 2nd smoothing image；Base
The mechanism of the gain-boosted op amp of each pixel is carried out to the image of the 1st viewpoint direction in the 1st gain profiles data；Based on the 2nd gain point
Cloth data carry out the mechanism of the gain-boosted op amp of each pixel to the image of the 2nd viewpoint direction；The 1st viewpoint after gain-boosted op amp will be carried out
The image in direction merges and as the anaglyph for stereoscopic display with the image for carrying out the 2nd viewpoint direction after gain-boosted op amp
And the mechanism for exporting；And by making the variable anaglyph to adjust for stereoscopic display of filtering diameter of smoothing wave filter
Between parallax amount mechanism, make filtering diameter contract when parallax is eliminated to zero, when parallax is preserved and two of input picture
The maximum disparity amount that can be produced between parallax pixel is same degree or is extended to more than it.
The image processing apparatus of the 3rd mode of the present invention are characterised by having：Image data acquisition part, it obtains logical
Corresponding to the 1st view data corresponding to the 1st viewpoint crossed the common optical system of at least a portion and obtain and the 2nd viewpoint
The 2nd view data；Smoothed data generating unit, it, will be by the 1st image respectively to the 1st view data and the 2nd view data
The picture of data and the 2nd pictorial data representation is respectively relative to the mode of parallax directions smoothing and is filtered, and thus generates the 1st flat
Cunningization data and the 2nd smoothed data；Destination image data generating unit, it uses the 1st smoothed data and the 2nd smoothed data
The geometric average of each corresponding pixel value is generating destination image data；And gain-boosted op amp portion, it uses target image
Data correct the 1st view data relative to the ratio of the 1st smoothed data.
The camera head of the 4th mode of the present invention is the dress of the shooting with imaging apparatus and above-mentioned image processing apparatus
Put, the 1st view data and the 2nd view data are based on the output of imaging apparatus and generate.
The image processing program of the 5th mode of the present invention makes computer perform following steps：Image data acquisition step, obtains
Take the 1st view data corresponding to the 1st viewpoint obtained by the common optical system of at least a portion and the 2nd viewpoint institute
Corresponding 2nd view data；Smoothed data generation step, respectively to the 1st view data and the 2nd view data, will be by the 1st
The picture of view data and the 2nd pictorial data representation is respectively relative to the mode of parallax directions smoothing and is filtered, and thus generates
1st smoothed data and the 2nd smoothed data；Destination image data generation step, it is smooth using the 1st smoothed data and the 2nd
Change the geometric average of each pixel value corresponding to data generating destination image data；And gain-boosted op amp step, using mesh
Logo image data correct the 1st view data relative to the ratio of the 1st smoothed data.
The image processing apparatus of the 6th mode of the present invention are characterised by having：Input is by an optical system to complete
The subject of the segment beam passed through from region different from each other in portion's light beam the 1st viewpoint side as obtained from being shot
To image and the 2nd viewpoint direction image at least two anaglyphs mechanism；Image and the 2nd to the 1st viewpoint direction
Multiple pictures that the image of viewpoint direction will be arranged respectively on the y directions orthogonal with the x directions of the 1st viewpoint of connection and the 2nd viewpoint
The pixel value of element is added, and generates the one-dimensional distributed data and the 2nd viewpoint direction of the 1st viewpoint direction after One Dimensional Projection respectively
The mechanism of one-dimensional distributed data；The one-dimensional distributed data of the 1st viewpoint direction after to One Dimensional Projection and the 2nd viewpoint direction it is one-dimensional
Distributed data applies respectively one-dimensional smoothing wave filter, and the 1st one-dimensional smoothing distributed data and the 2nd one-dimensional is thus generated respectively
The mechanism of smoothing distributed data；It is obtained based on the 1st one-dimensional smoothing distributed data and the 2nd one-dimensional smoothing distributed data
Between point and generate the mechanism of one-dimensional benchmark distributed data；By the way that the value of one-dimensional benchmark distributed data is one-dimensional smooth divided by the 1st
Change the value of distributed data to generate the mechanism of the 1st one-dimensional gain profiles data；Image to the 1st viewpoint direction, by the 1st one-dimensional increasing
Beneficial distributed data is commonly used with each y-coordinate, and carries out the mechanism of the gain-boosted op amp of the one-dimensional square respectively；And will carry out
The mechanism that the image of the 1st viewpoint direction after gain-boosted op amp is exported as the anaglyph for stereoscopic display, one-dimensional is flat
Cunningization wave filter is the maximum disparity amount that can be produced between the image than the 1st viewpoint direction and the image of the 2nd viewpoint direction
The wave filter smoothed in big scope.
The image processing apparatus of the 7th mode of the present invention are characterised by having：Image data acquisition part, it obtains logical
Corresponding to the 1st view data corresponding to the 1st viewpoint crossed the common optical system of at least a portion and obtain and the 2nd viewpoint
The 2nd view data；One Dimensional Projection data generating section, its respectively to the 1st view data and the 2nd view data, will with parallax
The pixel value of the multiple pixels arranged on the orthogonal direction in direction is added, and thus generates the 1st One Dimensional Projection data and the 2nd one-dimensional throwing
Shadow data；One-dimensional smoothed data generating unit, it to the 1st One Dimensional Projection data and the 2nd One Dimensional Projection data, is regarded respectively with maximum
Amplitude of samples more than residual quantity calculates rolling average, thus generates the 1st one-dimensional smoothed data and the 2nd one-dimensional smoothing number
According to；One-dimensional reference data generating unit, it is based on the 1st one-dimensional smoothed data and the 2nd one-dimensional smoothed data to generate one-dimensional base
Quasi- data；And shading value correction portion, it uses one-dimensional reference data relative to the figure of comparison the 1st of the 1st one-dimensional smoothed data
As data carry out shading value amendment.
The camera head of the 8th mode of the present invention is the dress of the shooting with imaging apparatus and above-mentioned image processing apparatus
Put, the 1st view data and the 2nd view data are based on the output of imaging apparatus and generate.
The image processing program of the 9th mode of the present invention is characterised by, makes computer perform following steps：View data
Obtaining step, obtain the 1st view data corresponding to the 1st viewpoint obtained by the common optical system of at least a portion,
With the 2nd view data corresponding to the 2nd viewpoint；One Dimensional Projection data genaration step, respectively to the 1st view data and the 2nd image
Data, the pixel value of the multiple pixels arranged on the direction orthogonal with parallax directions is added, and thus generates the 1st One Dimensional Projection
Data and the 2nd One Dimensional Projection data；One-dimensional smoothed data generation step, respectively to the 1st One Dimensional Projection data and the 2nd one-dimensional throwing
Shadow data, with amplitude of samples more than maximum disparity amount rolling average is calculated, and thus generates the 1st one-dimensional smoothed data and the
2 one-dimensional smoothed datas；One-dimensional reference data generation step, based on the 1st one-dimensional smoothed data and the 2nd one-dimensional smoothed data
To generate one-dimensional reference data；Shading value amendment step, using one-dimensional reference data relative to the 1st one-dimensional smoothed data ratio
Shading value amendment is carried out to the 1st view data.
Additionally, the summary of above-mentioned invention does not enumerate whole essential features of the present invention.In addition, the son of these feature groups
Combination can also become invention.
Description of the drawings
Fig. 1 is the figure of the structure of the digital camera for illustrating embodiments of the present invention.
Fig. 2 is the figure of the cross section structure for illustrating imaging apparatus.
Fig. 3 is the figure for illustrating the concept for defocusing in no-parallax pixel.
Fig. 4 is the figure for illustrating the concept for defocusing in parallax pixel.
Fig. 5 is the figure of the light intensity distributions for representing no-parallax pixel and parallax pixel.
Fig. 6 is to illustrate that the species of parallax pixel is the figure of the opening shape of the opening portion 104 in the case of two kinds.
Fig. 7 is the figure for illustrating fuzzy asymmetry.
Fig. 8 is the figure for representing anaglyph and ignoring relation between difference image and the depth of field.
Fig. 9 is the figure of the fuzzy asymmetry of explanation.
Figure 10 is the point for representing the object point for being present in non-focusing area as the figure of distribution.
Figure 11 is the figure of the arrangement of the real space for being denoted as and k-space.
Figure 12 is the figure for illustrating multiple resolution conversion.
Figure 13 is the figure of the structure of the digital camera for illustrating embodiments of the present invention.
Figure 14 is the figure of that illustrates shading value amendment.
Figure 15 is the figure for illustrating Function Fitting.
Specific embodiment
Hereinafter, by the embodiment explanation present invention for inventing, but following embodiment does not limit claims institute
The invention being related to.In addition, the combinations of features illustrated in embodiment need not be entirely necessary to the solution of invention.
[the 1st embodiment]
It is configured to lead to for a scene as the digital camera of the present embodiment of a mode of camera head
Cross the image for once shooting and generating multiple viewpoint numbers.Viewpoint each image different from each other is referred to as into anaglyph.In this reality
In applying mode, especially to generating the feelings for being based on the right anaglyph and left view difference image of two viewpoints corresponding with right eye and left eye
Condition is illustrated.Concrete condition will be described later, and the digital camera of present embodiment also can generate base together with anaglyph
In the viewpoint as reference direction it is central viewpoint, ignore difference image without parallax.
Fig. 1 is the figure of the structure of the digital camera 10 for illustrating embodiments of the present invention.There is digital camera 10 conduct to take the photograph
The photographic lens 20 of shadow optical system, will guide along the subject light beam of the incidence of optical axis 21 to imaging apparatus 100.Photography is saturating
Mirror 20 can be can be relative to the changeable type lens of the assembly and disassembly of digital camera 10.Digital camera 10 has imaging apparatus 100, control
Portion processed 201, A/D change-over circuits 202, memory 203, drive division 204, image processing part 205, storage card IF207, operating portion
208th, display part 209 and LCD drive circuits 210.
Additionally, as illustrated, the direction parallel with the optical axis 21 towards imaging apparatus 100 is set to into+Z-direction, will be
It is set to+X-direction towards the direction in paper in plane orthogonal to Z-axis ,+Y-axis side will be set to towards the direction above paper
To.It is that X-axis is horizontal direction with the relation found a view to photograph, Y-axis is vertical direction.In some figures afterwards, by the seat of Fig. 1
Parameter as benchmark displaing coordinate axle, so as to the direction of clearly each figure.
Photographic lens 20 is made up of multiple optical lens groups, makes the subject light beam from scene near its focus face
Imaging.Additionally, in FIG, for convenience of description, taken the photograph to represent as representative with being configured in the imaginary 1 piece of lens near pupil
Shadow lens 20.
Imaging apparatus 100 is configured near the focus face of photographic lens 20.Imaging apparatus 100 is to be configured with two dimensions
Such as CCD, cmos sensor etc. imageing sensor of multiple photo-electric conversion elements.Imaging apparatus 100 is by the timing control of drive division 204
System, the subject picture being imaged on sensitive surface is converted into picture signal and is exported to A/D change-over circuits 202.To A/D conversions
Picture signal of the picture signal of the output of circuit 202 comprising left view point and right viewpoint.
The picture signal that imaging apparatus 100 is exported is converted into data image signal and to memory by A/D change-over circuits 202
203 outputs.Memory 203 is implemented various image procossings as working space by image processing part 205, so as to generate picture number
According to.Especially, image processing part 205 has image data generating section 230, smoothed data generating unit 231, destination image data
Generating unit 232 and gain-boosted op amp portion 233.
Image data generating section 230 is generated corresponding with left view point using left view point and the data image signal of right viewpoint
Left parallax image data and right parallax image data corresponding with right viewpoint.With regard to left parallax image data and right anaglyph number
According to the details of generation will be described later.
Smoothed data generating unit 231, respectively will be poor by left view for left parallax image data and right parallax image data
The picture that view data and right parallax image data are represented was carried out in the way of relative to left and right directions, i.e. parallax directions smoothing
Filter.Thus, generate by the left smoothed data after left view difference image data smoothing and by right parallax image data after smoothing
Right smoothed data.To be described later with regard to the details of the generation of left smoothed data and right smoothed data.
Destination image data generating unit 232 uses each pixel value corresponding to left smoothed data and right smoothed data
Geometric average generating destination image data.The details of the generation of destination image data will be described later.
Left view difference image number is corrected using destination image data relative to the ratio of left smoothed data in gain-boosted op amp portion 233
According to.And using destination image data relative to the ratio of right smoothed data correcting right parallax image data.
Image processing part 205 also undertakes according to selected picture format to adjust the image of view data etc. in addition
The general function of process.The view data for being generated is converted into showing signal by LCD drive circuits 210, and is displayed in aobvious
Show in portion 209.In addition, record is in the storage card 220 of storage card IF207 is installed on.
A series of photographic process accepts the operation of user and to the output function of control unit 201 letter by operating portion 208
Number and start.The various actions such as AF, the AE with photographic process are controlled by control unit 201 and performed.
Next, one of the structure of explanation imaging apparatus 100.Fig. 2 is the outline in the section for representing imaging apparatus 100
Figure.
Imaging apparatus 100 is arranged with order lenticule 101, colored filter 102, aperture mask from object side
103rd, wiring layer 105 and photo-electric conversion element 108 and constitute.Photo-electric conversion element 108 is converted into electric signal by by incident light
Photodiode constitute.Photo-electric conversion element 108 is arranged with two dimensions multiple on the surface of substrate 109.
The Jing such as the picture signal, the control signal of control photo-electric conversion element 108 that are converted to by photo-electric conversion element 108
Received and dispatched by the wiring 106 located at wiring layer 105.In addition, have arrange correspondingly with each photo-electric conversion element 108 and
In two dimensions the aperture mask 103 of the opening portion 104 of repeated arrangement, with wiring layer 105 setting is mutually grounded.It is open as described later
The displacement by corresponding photo-electric conversion element 108 of portion 104, but relative position is strictly determined.Details will be described later,
By the effect of the aperture mask 103 with the opening portion 104, in the subject light beam that photo-electric conversion element 108 is received
Middle generation parallax.
On the other hand, on the photo-electric conversion element 108 for not producing parallax, there is no aperture mask 103.In other words,
Can be described as being provided with possess do not limit subject light beam relative to the incidence of corresponding photo-electric conversion element 108, i.e., make incidence
Light beam entirely through opening portion 104 aperture mask 103.Although parallax will not be produced, substantially formed by wiring 106
The incident subject light beam of the regulation of opening 107, thus, it is also possible to wiring 106 being regarded as, will not produce parallax makes incidence
The overall aperture mask of light beam.Aperture mask 103 accordingly can independently of one another be arranged with each photo-electric conversion element 108,
Can in the lump be formed relative to multiple photo-electric conversion elements 108 in the same manner as the manufacturing process of colored filter 102.
Colored filter 102 is located on aperture mask 103.Colored filter 102 be to each photo-electric conversion element 108 so that
The mode that specific wave band is passed through colours the optical filter that obtain and photo-electric conversion element 108 is arranged correspondingly respectively.It is right
In output coloured image, if at least two colored filters different from each other are arranged, but for acquisition more high image quality
Coloured image, arranges more than three kinds of colored filter preferably.For example the red of red band transmission can be made with clathrate arrangement
Colo(u)r filter (R optical filters), makes the green band green color filter (G optical filters) for passing through and the indigo plant for making blue wave band transmission
Colo(u)r filter (B optical filters).Colored filter not only can be the complementary colo(u)r filter of the combination of primary colors RGB, or YCM
Combination.
Lenticule 101 is located on colored filter 102.Lenticule 101 is for the subject light beam of incidence is more
Collector lens of the ground to the guiding of photo-electric conversion element 108.Lenticule 101 sets correspondingly with each photo-electric conversion element 108
Put.With regard to lenticule 101, it is considered preferred to the pupil centre of photographic lens 20 and the relative position relation of photo-electric conversion element 108,
Its light axial displacement is made in the way of making more subject light beams be directed into photo-electric conversion element 108.Furthermore, it is also possible to
The position of the opening portion 104 of aperture mask 103 adjusts together its allocation position, so that specific subject light beam described later
It is more incident.
By the aperture mask 103, the colored filter 102 that so arrange correspondingly with each photo-electric conversion element 108
And a unit of lenticule 101 is referred to as pixel.Especially the pixel for being provided with the aperture mask 103 for producing parallax is referred to as into parallax picture
Element, the pixel that will be not provided with the aperture mask 103 of generation parallax is referred to as no-parallax pixel.There is also the parallax picture of left view point
The feelings that element is designated as parallax Lt pixels, the parallax pixel of right viewpoint is designated as parallax Rt pixels, no-parallax pixel is designated as N pixels
Condition.In addition, there is also and the anaglyph of left view point is designated as into parallax Lt images, the anaglyph of right viewpoint is designated as parallax Rt figures
As, difference image will be ignored be designated as the situation of N images.For example, the effective pixel area in imaging apparatus 100 is that 24mm × 16mm is left
In the case of the right side, pixel count reaches 12,000,000 or so.
Additionally, in the case of the high imageing sensor of light gathering efficiency, photoelectric transformation efficiency, it is also possible to be not provided with lenticule
101.In addition, overleaf in the case of irradiation type imageing sensor, wiring layer 105 is arranged on contrary with photo-electric conversion element 108
Side.If in addition, the opening portion 104 of aperture mask 103 have color component, also can by colored filter 102 with open
Mouth mask 103 is integrally formed.Additionally, in the case of output black-and-white image signal, being not provided with colored filter 102.
In addition, in the present embodiment, although arrange 106 splits of aperture mask 103 and wiring, but it is also possible to by cloth
Line 106 undertakes the function of the aperture mask 103 in parallax pixel.That is, the opening shape of regulation, and root are formed by wiring 106
Incident beam is limited according to the opening shape and only guide specific segment beam to photo-electric conversion element 108.In this case, it is excellent
Form slection most leans on the side of photo-electric conversion element 108 into the wiring 106 of opening shape in wiring layer 105.
In addition, aperture mask 103 can also be formed by the transmission block film overlappingly arranged with photo-electric conversion element 108.
In this case, aperture mask 103 is for example by SiN film and SiO2Film stacks gradually and is used as passing through block film, and will be equivalent to out
The region of oral area 104 is removed by etching and is formed.
＜ parallax pixels and fuzzy behaviour ＞
Next, explanation parallax Lt pixels and parallax Rt pixel light in the case of the concept for defocusing.First, simply
Illustrate the concept for defocusing in no-parallax pixel.Fig. 3 is the figure for illustrating the concept for defocusing in no-parallax pixel.Such as (a) of Fig. 3
It is shown, in the case where the object point as subject is present in focal position, pass through and reach imaging apparatus from lens pupil
The subject light beam of sensitive surface shows precipitous light intensity distributions centered on the pixel of corresponding picture point.That is, if receiving
The no-parallax pixel of the efficient beam entirety passed through from lens pupil is arranged near picture point, then pixel corresponding with picture point is defeated
Go out value maximum, the output valve for being arranged in the pixel of periphery is drastically reduced.
On the other hand, shown in (b) of such as Fig. 3, if object point on the direction away from imaging apparatus sensitive surface from focal position
Skew, then compared with the situation that object point is present in focal position, subject light beam shows to put down on imaging apparatus sensitive surface
Slow light intensity distributions.That is, show on the basis of the output valve reduction in the pixel of corresponding picture point, up to more periphery
The pixel still distribution with output valve.
As shown in (c) of Fig. 3, if object point further offsets from focal position, subject light beam is received in imaging apparatus
Gentler light intensity distributions are shown on light face.That is, the output valve shown in the pixel of corresponding picture point further drops
On the basis of low, until the pixel still distribution with output valve of more periphery.
As shown in (d) of Fig. 3, in the situation that object point offsets on the direction of close imaging apparatus sensitive surface from focal position
Under, also show the situation identical light intensity distributions offset up in the side away from imaging apparatus sensitive surface with object point.
Fig. 4 is the figure for illustrating the concept for defocusing in parallax pixel.Parallax Lt pixels and parallax Rt pixels receive from conduct
The subregion of lens pupil and respectively to light axis target setting two parallax hypothetical pupils in some reach it is being shot
Object beam.In this manual, the quilt that will be reached by the hypothetical pupil different from each other received from single lens pupil
The mode taken the photograph object beam and shot to anaglyph is referred to as simple eye pupil cutting shooting mode.
As shown in (a) of Fig. 4, in the case where the object point as subject is present in focal position, no matter from which
The subject light beam that parallax hypothetical pupil passes through, shows precipitous luminous intensity centered on the pixel of corresponding picture point point
Cloth.If parallax Lt pixel arrangements are near picture point, the output valve of pixel corresponding with picture point is maximum, is arranged in the pixel of periphery
Output valve drastically reduce.In addition, even if parallax Rt pixel arrangements are near picture point, the output valve of pixel corresponding with picture point
Maximum, the output valve for being arranged in the pixel of periphery is also drastically reduced.That is, no matter subject light beam is from which parallax hypothetical pupil
Pass through, the output valve of the output valve maximum and the pixel for being arranged in periphery that show pixel corresponding with picture point is drastically reduced
Distribution, each distribution is consistent with each other.
On the other hand, shown in (b) of such as Fig. 4, if object point on the direction away from imaging apparatus sensitive surface from focal position
Skew, then compared with the situation that object point is present in focal position, the peak value of the light intensity distributions that parallax Lt pixels are shown shows
Now in positioning away to a direction from pixel corresponding with picture point, and its output valve is reduced.In addition, the picture with output valve
The amplitude of element also expands.That is, due to the expansion that the horizontal direction tool relative to imaging apparatus sensitive surface is somewhat like, so fuzzy quantity
Increase.The peak value of the light intensity distributions that parallax Rt pixels are shown is revealed in from pixel corresponding with picture point to parallax Lt pixels
In the direction in opposite direction with and equidistantly position away from, similarly its output valve is reduced.In addition, similarly having
The amplitude for having the pixel of output valve also expands.That is, gentle same light is become compared with the situation that object point is present in focal position
Intensity distribution is manifested each other with equidistant separation.The peak of the light intensity distributions that parallax Lt pixels and parallax Rt pixels are shown
Side-play amount between value is equivalent to parallax amount.
In addition, as shown in (c) of Fig. 4, if object point further offsets from focal position, the state phase with (b) of Fig. 4
Than the same light intensity distributions for further becoming gentle more discretely manifest.Because the expansion of a picture becomes much larger, so fuzzy
Amount increases.Further, since the skew between the peak value of light intensity distributions that shown of parallax Lt pixels and parallax Rt pixels also becomes
Greatly, so parallax amount also increases.That is, it may be said that object point offsets bigger from focal position, fuzzy quantity and parallax amount are got over
Increase.
As shown in (d) of Fig. 4, in the situation that object point offsets on the direction of close imaging apparatus sensitive surface from focal position
Under, with the state of (c) of Fig. 4 conversely, the peak value of light intensity distributions that parallax Rt pixels are shown be revealed in from picture point pair
The pixel answered to said one direction positions away from.The peak value of the light intensity distributions that parallax Lt pixels are shown be revealed in
The direction in opposite direction with one in parallax Rt pixels positions away from.I.e., with the offset direction of object point correspondingly, it is determined that regarding
The peak value of the light intensity distributions that difference Lt pixels and parallax Rt pixels are shown is revealed in from pixel corresponding with picture point to which
Direction positions away from.
When will in the change of light intensity distributions illustrated in fig. 3 and Fig. 4 illustrate light intensity distributions change be respectively prepared song
During line chart, as shown in Figure 5.Fig. 5 is the figure of the light intensity distributions for representing no-parallax pixel and parallax pixel.In figure, transverse axis is represented
Location of pixels, center is location of pixels corresponding with picture point.The longitudinal axis represents the output valve of each pixel, due to the output valve reality
It is directly proportional to luminous intensity in matter, so illustrating as luminous intensity in figure.
Additionally, as described above, no matter object point on the direction of close imaging apparatus sensitive surface from the feelings of focal position skew
The situation that condition or object point offset up in the side away from imaging apparatus sensitive surface, shows identical light intensity distributions, because
This, eliminates the change of the light intensity distributions in the case where the side of close imaging apparatus sensitive surface offsets up in figure.Close
Parallax Lt pixels and parallax Rt in the case that object point offsets on the direction of close imaging apparatus sensitive surface from focal position
The peak value of the light intensity distributions that pixel is shown, due to the feelings offset up in the side away from imaging apparatus sensitive surface with object point
The peak value of the light intensity distributions that the parallax Lt pixels and parallax Rt pixels under condition is shown is identical, therefore has been also carried out omitting.
(a) of Fig. 5 is the curve map of the change for representing light intensity distributions illustrated in fig. 3.Distribution curve 1801 represent with
The corresponding light intensity distributions of (a) of Fig. 3, illustrate most precipitous situation.Distribution curve 1802 represents light corresponding with (b) of Fig. 3
Intensity distribution, in addition, distribution curve 1803 illustrates light intensity distributions corresponding with (c) of Fig. 3.Compared with distribution curve 1801,
Peak value is gradually reduced, it is known that with the situation for expanding.
(b) of Fig. 5 is the curve map of the change for representing light intensity distributions illustrated in fig. 4.Distribution curve 1804 and distribution
Curve 1805 represents respectively the light intensity distributions of the parallax Lt pixels of (b) of Fig. 4 and the light intensity distributions of parallax Rt pixels.From figure
Understand, these distributions are relative to center into the symmetrical shape of line.In addition, the synthesis point that these distribution curve superpositions are obtained
Cloth curve 1806 illustrates that the distribution curve 1802 of (b) to (b) relative to Fig. 4 in Fig. 3 of equal defocus condition is similar
Shape.
Distribution curve 1807 and distribution curve 1808 represent respectively the parallax Lt pixels of (c) of Fig. 4 light intensity distributions and
The light intensity distributions of parallax Rt pixels.Knowable to figure, these distributions are also relative to center into the symmetrical shape of line.In addition,
The synthesis distribution curve 1809 that these distribution curve superpositions are obtained is illustrated and be in equal defocus condition with (c) relative to Fig. 4
Fig. 3 (c) the similar shape of distribution curve 1803.Additionally, the light intensity distributions of the parallax Lt pixels of (d) of Fig. 4 and regarding
The light intensity distributions of difference Rt pixels are the luminous intensity by the light intensity distributions of the parallax Lt pixels of (c) of Fig. 4 and parallax Rt pixels
The relation that the position of distribution is exchanged, therefore it is respectively equivalent to distribution curve 1808 and distribution curve 1807.
Fig. 6 is to illustrate that the species of parallax pixel is the figure of the opening shape of the opening portion 104 in the case of two kinds.Fig. 6's
(a) show the shape of the opening portion 104r of the shape of the opening portion 104l of parallax Lt pixels and parallax Rt pixels with by no parallax
Each shape identical example that the shape of the opening portion 104n of pixel is obtained with the segmentation of center line 322.That is, in Fig. 6
(a) in, the area of the opening portion 104n of no-parallax pixel for the opening portion 104l of parallax Lt pixels area and parallax Rt pictures
The area sum of the opening portion 104r of element.In the present embodiment, the opening portion 104n of no-parallax pixel is referred to as into open
Opening portion, by opening portion 104l and opening portion 104r the opening portion of half opening is referred to as.In opening in photo-electric conversion element
In the case of central authorities, referred to as the opening portion is towards reference direction.The opening portion 104l of parallax Lt pixels and opening for parallax Rt pixels
Oral area 104r is relative to the imaginary center line passed through from the center (pixel center) of each self-corresponding photo-electric conversion element 108
322, off normal round about each other.Therefore, the opening portion 104r of the opening portion 104l of parallax Lt pixels and parallax Rt pixels point
Not a direction relative to center line 322 and with an other direction in opposite direction on produce parallax.
(b) of Fig. 6 illustrates that object point is being received away from imaging apparatus in the pixel of each opening portion shown in (a) with Fig. 6
From the light intensity distributions in the case of the skew of focal position on the direction in light face.In figure, transverse axis represents location of pixels, centre bit
It is location of pixels corresponding with picture point to put.In addition, the distribution curve 1804 of (b) of curve Lt and Fig. 5 is quite, curve Rt and Fig. 5
(b) distribution curve 1805 it is suitable.Curve N is corresponding with no-parallax pixel, illustrates the synthesis distribution curve with (b) of Fig. 5
1806 similar shapes.In addition, each opening portion 104n, opening portion 104l, opening portion 104r play the work(as opening aperture
Energy.Therefore, with the mould that area is opening portion 104l (opening portion 104r) no-parallax pixels of the opening portion 104n of area at double
Paste amplitude with it is shown in the synthesis distribution curve 1806 by (b) of Fig. 5, parallax Lt pixels and parallax Rt pixels be superimposed and obtained
Curve fuzzy amplitude be same degree.
(c) of Fig. 6 illustrates that object point is received in close imaging apparatus in the pixel of each opening portion shown in (a) with Fig. 6
From the light intensity distributions in the case of the skew of focal position on the direction in light face.In figure, transverse axis represents location of pixels, centre bit
It is set to location of pixels corresponding with picture point.Curve Lt, the curve Rt of (c) of Fig. 6 maintains the no parallax picture with opening portion 104n
The fuzzy amplitude of element and pass of the fuzzy amplitude for same degree that parallax Lt pixels and parallax Rt pixels are superimposed the curve for obtaining
System, the curve Lt, curve Rt relative to (b) of Fig. 6 and position relationship is contrary.
The ＜ depth of field and asymmetric fuzzy ＞
Next, the relation between the explanation depth of field and fuzzy asymmetry.Also can learn from (b), (c) of Fig. 6, non-
In focusing area, the fuzzy amplitude of the fuzzy Amplitude Ratio no-parallax pixel of parallax pixel is narrow.This means due to (a) of Fig. 6
The aperture mask of parallax pixel and make the incident beam of lens substantially be concentrated as right half part and left-half.In other words,
It may be said that generating two hypothetical pupils in left and right in single lens pupil.That is, the aperture area of the aperture mask of parallax pixel
Play and acted on the effect identical of lens stop.
Generally, the deep image of the depth of field is shot if reducing glass aperture.Aperture mask in parallax pixel is opened
It is mouthful short in the horizontal direction and long in vertical direction.Therefore, it is being shot with frequency content in the horizontal direction to ordinate etc.
Object shoots the deep image of the depth of field, and in contrast, to horizontal line etc., in vertical direction the subject with frequency content shoots
The shallow image of the depth of field.
Fig. 7 is the figure for illustrating fuzzy asymmetry.For example, the foursquare fritter as (a) to Fig. 7
(patch) when subject is shot, in focusing area, subject picture as (a) of Fig. 7 is obtained.Fig. 7's
B in (), the subject picture that left parallax pixel and right parallax pixel are captured overlaps and illustrates.In non-focusing area,
Photograph the horizontal direction as shown in (b) of Fig. 7 it is fuzzy less, ordinate seem the object of sharp steep (sharp) than horizontal line
Body image.That is, because the opening of the aperture mask in parallax pixel is in the horizontal direction and the vertical direction into asymmetry, so
Subject picture horizontally and vertically on obscure as asymmetric.It can also be referred to as fuzzy anisotropy.
If the subject picture coincidence of the subject picture of the left eye of (b) of Fig. 7 and right eye is shown from 3D
Image obtains 2D images, then can produce in 2D images because the sharp steep fuzzy and caused two-wire in horizontal direction is fuzzy such
, fuzzy ((c) of Fig. 7) being less expected to.It is therefore preferable that can realize show 3D rendering by image procossing
The fuzzy asymmetry of the anaglyph of left and right is reduced and obtains naturally fuzzy such amendment, or is shown in 2D images
In be not in two-wire is fuzzy such, obtain natural fringe amendment.
Fig. 8 is the figure for representing anaglyph and ignoring relation between difference image and the depth of field.Specifically, Fig. 8 is to represent
The pel spacing of imaging apparatus 100 is set to into a [mm], the bar graph to frequency for subject picture as f [bar/mm] to shoot
When ordinate bar graph and horizontal line bar graph when being rotated by 90 ° and being shot MTF (Modulation Transfer
Function：Modulation transfer function) characteristic object distance interdependence figure.The longitudinal axis represents MTF, and transverse axis is represented from number
Camera 10 apart from d.MTF near the optical axis by focusing position is set in the case of 1, if MTF distributions are represented bar graph
Then how to decay from focusing position is movable.(a) of Fig. 8 represents the quilt of the fixed frequency ignored in difference image (N images)
Take the photograph the ordinate bar graph related to object distance of object picture and the MTF distributions of horizontal line bar graph.As shown in (a) of Fig. 8,
In difference image is ignored, ordinate bar graph is consistent with the MTF of horizontal line bar graph distributions.(b) of Fig. 8 represents anaglyph (parallax
Lt images and parallax Rt images) in fixed frequency subject picture the ordinate bar graph related to object distance and
The MTF distributions of horizontal line bar graph.The MTF distributions of the horizontal line bar graph shown in (b) of Fig. 8 and the horizontal line striped shown in (a) of Fig. 8
The MTF distributions of figure are consistent.On the other hand, the MTF distributions of the ordinate bar graph shown in (b) of Fig. 8 and indulging shown in (a) of Fig. 8
The MTF distributions of line bar graph are compared, and the high interval of contrast is distributed extensively, can read the depth of field for deep this case.In other words,
If making bar graph movable from focusing position, in anaglyph, the contrast in horizontal line bar graph and horizontal line bar graph
It is different.The fuzzy asymmetry that its generation previously figure 7 illustrates.
As the intrinsic phenomenon produced in simple eye pupil cutting shooting mode, enumerate in WO2012/039346 in left and right
Anaglyph between produce the problem of different shading values.But, in simple eye pupil cutting shooting mode, as with it is common
2D image photographies compare the problem having to take into account that, in addition to shading value, also there is fuzzy characteristic between left and right for non-right
The problem of title.Details will be described later, for example, the quadrangle as by Macbeth color table (macbeth chart)
Fritter faintly shoot in the case of, as described above, subject picture horizontally and vertically on obscure as non-
Symmetrically, moreover, obscure as asymmetric at the left and right two ends of anaglyph.Specifically, in left view difference image, a side's
Border thickens, and produces in the border of the opposing party seem sharp steep such fuzzy in contrast.In right anaglyph
The phenomenon is contrary.The fuzzy left and right asymmetry is recognized as two-wire and obscures, and can bring unnatural to stereo-picture.
Labor one-eyed stereo camera system, its result understand shading value problem, fuzzy left and right asymmetry and
There is extremely close relation between parallax amount.Therefore, in left view difference image and right anaglyph, pair related to this three
Part be modified in the case of, other characteristics can be brought and significantly affect.Can its result be, indefinite to actually generate certainly
Right stereo-picture.For example, in above-mentioned WO2012/039346, without shading value amendment disclosed in detail and parallax amount
, there is the indefinite situation that can actually generate stereo-picture in relation.As for fuzzy behaviour then all without description.In view of these shapes
Condition, in the present embodiment, specifies the relation of above-mentioned three, illustrates the generation technique for seeming natural stereo-picture.
Left and right asymmetry and its amendment ＞ that ＜ is obscured
Because parallax pixel receives to be projected on the incident beam of hypothetical pupil, so the shape of hypothetical pupil is revealed as obscuring
Characteristic.Parallax pixel plays a part of the light beam of the unilateral half for substantially making the light passed through from lens and passes through.The light of lens
Pupil is that circle, therefore hypothetical pupil are semicircle.Therefore, between the anaglyph of left and right, fuzzy being shaped as is asymmetric.
Fig. 9 is the figure of the fuzzy asymmetry of explanation.As shown in figure 9, in left parallax pixel, with semicircular arc side phase
Corresponding obscuring can rightly show, and in contrast, obscure corresponding with semicircular disconnection side cannot be showed rightly.
I.e. it is easy to be changed into sharp steep edge and obscure few.Be also in right parallax pixel similarly, it is relative with semicircular arc side contacts
Answer it is fuzzy can rightly show, it is in contrast, corresponding with semicircular disconnection side fuzzy to be easily changed into sharp steep side
Edge and it is fuzzy few.But, in left parallax pixel and right parallax pixel, the position of arc is contrary.For above situation, above-mentioned is double
Line blooming more significantly manifests.It is asymmetric fuzzy for the such left and right of amendment, as shown in figure 9, expecting left view
The fuzzy of presence opposite characteristic that difference pixel and right parallax pixel are captured is mutually mixed, and makes between left and right to obscure as far as possible
Symmetrization.
As the scheme to this, generate the mean chart after the fuzzy hybrid of the fuzzy and right anaglyph of left view difference image
Picture, and as target image carrying out the asymmetric fuzzy revising in fuzzy left and right.Such as in the " nothing of the 3rd embodiment described later
In the step of generation of the benchmark image of parallax " explanatorily, if only merely generating left view difference image and right anaglyph
The average image, then can produce the phenomenon that parallax shading value offsets and parallax is not completely eliminated.Therefore, if carrying out such
The amendment of the image of parallax as target will be eliminated, then can only generate the image for eliminating parallax.It becomes and stereo-picture
The contrary amendment of generation, it is therefore necessary to avoid.For the information for only extracting fuzzy left and right asymmetry, it is necessary to avoid repairing
Parallax component is included in positve term.Parallax component in order to eliminate correction term, needs to apply left view difference image and right anaglyph
Plus with the horizontal direction of parallax same degree on one-dimensional smoothing wave filter or by its along transverse and longitudinal expand two dimension smooth
Change wave filter.The place reason ＜ ＞ applied after the smoothing computing is represented.
Thereby, it is possible to realize the amendment of fuzzy left and right asymmetry.Its reason is, fuzzy for left view difference image
The difference of the characteristic of composition and the characteristic of the fuzzy composition of right anaglyph, even if implementing the significantly smoothing of parallax amount degree
Also can remaining after process.If the amplitude of samples of wave filter is excessively huge relative to parallax amount, fuzzy characteristic will not be shown
Difference, it is therefore desirable to be limited to parallax amount degree.If making the scope of the smoothing in the formula to be gradually reduced from parallax amount degree
And the mode that amplitude of samples is zero (i.e. without smoothing) is continuously changed, then parallax amount is captured from anaglyph
Initial parallax amount continuously changes to parallax zero.Thus, it is also possible to pass through to smooth the filtering diameter of wave filter, sample
Amplitude is controlling parallax amount.
It is modified in the fuzzy corresponding mode with the geometric average of the anaglyph after smoothing in above formula, but is made
The viewpoint for taking arithmetic average is there is also for viewpoint.In this case, try research association become what kind of obscure.Amendment type is as follows.
Figure 10 is the point for representing the object point for being present in non-focusing area as the figure of distribution.In figure, transverse axis represents pixel position
Put, center is location of pixels corresponding with picture point.The longitudinal axis represents pixel value.In (a), (b) of Figure 10, the point of left view point
As the point for being distributed Lt and right viewpoint shows the feelings of the opening portion for half opening of the parallax pixel of the left and right shown in Fig. 6 as distribution Rt
Fuzzy amplitude under condition, in horizontal direction.
The point of the left view point of (a) of Figure 10 is as the point of distribution Lt and right viewpoint is as distribution Rt is by the function table of location of pixels x
Show.The point of left view point is as the point of distribution Lt and right viewpoint is as the arithmetic average of distribution Rt is also by the function representation of location of pixels x.Example
Such as, at location of pixels x1, Lt and Rt is respectively provided with pixel value, thus operation result be on the occasion of.At location of pixels x2, although Lt
There is no pixel value, but Rt has a pixel value, thus operation result be still on the occasion of.As described above, arithmetic average is being calculated
In the case of, as long as any one in Lt and Rt has pixel value, then its operation result be on the occasion of.Therefore, if as whole observation
The point of each pixel position is as distribution Lt and puts as the arithmetic average of distribution Rt, then be a picture distribution Lt as (a) of Figure 10 is shown
And the extended amplitude sum of Rt.Mould corresponding to its no-parallax pixel equivalent to the opening portion with the open shown in Fig. 6
Paste amplitude.
Additionally, arithmetic average is of the computing of the extended amplitude sum for acquisition point picture distribution Lt and Rt.As long as
An extended amplitude sum of picture distribution Lt and Rt is obtained in that, then can also utilize other computings.
The point of left view point is as the point of distribution Lt and right viewpoint is as the geometric average of distribution Rt is also by the function table of location of pixels x
Show.For example, at location of pixels x1, Lt and Rt is respectively provided with pixel value, thus operation result be on the occasion of.On the other hand, in pixel
At the x2 of position, although Rt has pixel value, Lt does not have pixel value, therefore operation result is 0.As described above, calculating
In the case of geometric average, in the case that any one in Lt and Rt has pixel value, its operation result be on the occasion of.Therefore,
If being distributed Lt as the point picture of each pixel position of whole observation and putting the geometric average as being distributed Rt, such as (b) institute of Figure 10
Show, be a product for the extended amplitude of picture distribution Lt and Rt.The extended amplitude of geometric average narrows compared with arithmetic average, if parallax
Amount increases and then further diminishes relatively, it can be said that its ratio has correlation with parallax amount.That is, following formula represents that parallax reflects
Penetrate.
Here, σ 1 is the constant to the conversion of absolute extended amplitude of picture, for arbitrary image takes fixed value.
Additionally, geometric average is of the computing of the product of the extended amplitude for acquisition point picture distribution Lt and Rt.As long as
A product for the extended amplitude of picture distribution Lt and Rt is obtained in that, then can also utilize other computings.Additionally, generally, due to can
Think that subject is the set of object point, so image also can be referred to as the integration of the point picture relative to each object point.Thus, energy
It is enough with above-mentioned point as distribution it is fuzzy in the same manner as consider the fuzzy of image.
As shown in Figure 10, the left and right of script anaglyph is fuzzy equal with the fuzzy amplitude of half opening, in geometric average
By the fuzzy direct transmission of half opening.By the fuzzy transmission of open in another aspect arithmetic average.If only expressing fuzzy amplitude
Transitive relation showing the formula of above-mentioned geometric average and the formula of arithmetic average, then become following formula.
In the case of geometric average
(half opening is obscured)=(half opening is obscured) × (half opening is obscured)/(half opening is obscured)
In the case of arithmetic average
(open is obscured)=(half opening is obscured) × (open is obscured)/(half opening is obscured)
Therefore, left view difference image and right anaglyph, with half opening in the region for generating parallax each other is in
It is fuzzy and in the case of the detached state of parallax, if carrying out the asymmetric fuzzy revising of left and right, detached with parallax
Under state, the fuzzy amplitude with original same degree is remained in the case of geometric average as before.On the other hand in arithmetic average
In the case of, it is fuzzy with open respectively in the state of separating with parallax.Although by left view difference image and right disparity map
Seem exactly 2D images as the image for obtaining that overlaps merely should work as, but according to arithmetic average, then two in horizontal direction
Individual open is fuzzy to be overlapped, and is transversely obscured and is expanded.Its result is to cause to implement excessive fuzzy revising.If on the other hand
Using geometric average, then because 2D images are only by the fuzzy overlap of half opening, so as overall it can be seen that scheming with common 2D
As identical, in machine and transverse direction open is isotropic fuzzy.Therefore, from from the viewpoint of fuzzy transmission, for a left side
Right asymmetric fuzzy revising, will take the fuzzy as in the case of target of the image that geometric average is obtained, using the teaching of the invention it is possible to provide preferably
Image.If assuming fuzzy as target using take that arithmetic average obtains, the not distinct of stagnant plate in the horizontal direction can be provided
Image, it should be avoided that.
Details will be described later, if correcting fuzzy left and right asymmetry as described above, also have actually
Also the subsidiary effect of the shading value of generation in 3D rendering can simultaneously be solved the problems, such as.
First, in the 1st embodiment, illustrate that the image for already generating stereo-picture to carrying out one-eyed stereo shooting is carried out
Because of the technology of left and right asymmetry amendment and shading value amendment fuzzy caused by parallax pixel.Then, in the 2nd embodiment,
Illustrate the situation for carrying out parallax amount regulation.Finally, in the 3rd embodiment, in order to the fuzzy left and right for expressing anaglyph it is non-right
Title property amendment, shading value amendment and parallax eliminate the relation between this three, illustrate to one-eyed stereo camera head by veneer
Sensor is while shoot the image that color and parallax are obtained and the technology for jointly restoring both color and parallax.
Embodiments ＞ of ＜ the 1st
In the present embodiment, the stereo-picture of black and white is processed.In the situation of the coloured image formed by the colors of RGB tri-
Under, as long as carrying out to assorted face and the process of black white image identical, therefore omit the description.The order of process is substantially as follows.
1) parallax image data input
2) generation of image is smoothed
3) generation of the benchmark image of 3D rendering amendment
4) generation of gain profiles data
5) the asymmetric fuzzy revising in the left and right of anaglyph and shading value amendment
Hereinafter, illustrate in order.
1) parallax image data input
It is input into and left view difference image Lt (x, y) and right anaglyph Rt (x, y) for obtaining is shot to common subject picture.This
When, each anaglyph is the linear gray level data for not carrying out gamma-corrected.In addition, in the case where coloured image is processed,
The RGB data being contemplated to be under the sensor light spectral sensitivity original state for not carrying out color treatments.
2) generation of image is smoothed
In the way of avoiding eliminating parallax because of the asymmetric fuzzy revising in left and right and shading value amendment, to left view difference image and
Right anaglyph applies in the horizontal direction and maximum disparity amount same degree or the smoothing wave filter bigger than its, enters and is about to regard
The process that difference information is completely eliminated.Consider one-dimensional smoothing wave filter and two dimension smoothing wave filter both of these case.
Maximum disparity amount between the anaglyph of left and right by the shooting area of imaging apparatus, pel spacing and optical system it
Between relationship specifications, be 5% or so of the horizontal direction order of magnitude (order) for image planes entirety.If horizontal direction is 2000
Pixel or so is then 100 pixels or so.Apply corresponding smoothing wave filter.By filter radius by σ0Represent, be set as
The order of magnitude of maximum disparity amount.It may be said that this is filtered into local smoothing method.Smoothing view data is protected with the precision of its locality
Hold fuzzy behaviour and shading value composition.
Additionally, maximum disparity amount is according to the focal length and f-number of long focus lens, standard lens, wide-angle lens etc. and not
Together, therefore by each optical condition σ is preset0Value preferably.Method with regard to obtaining from image itself, for example, modus ponens (1)
Histogram, the independent variable of histogrammic transverse axis for giving its maximum is set as into σ0Preferably.
In the case of Gaussian Blur, represented with following formula.In the sense that local average is taken, by left smoothing view data
Represented by ＜ L (x, y) ＞, ＜ R (x, y) ＞ respectively with right smoothing view data.
Gaussian blur filter
In the case of one-dimensional smoothing
In the case of two dimension smoothing
Even if additionally, Gaussian Blur can not be simply used, it is also possible to what is all represented by the positive filter factor of identical
Uniform wave filter is alternative.
3) generation of the benchmark image of 3D rendering amendment
From eliminating the left smoothing image of parallax and eliminating the right smoothing image of parallax, generate for representing fuzzy
The benchmark image of the 3D rendering amendment of the target of amendment and the reference grade of shading value amendment.It is represented by ＜ M (x) ＞.
With regard to the generation method of the datum mark, from from the viewpoint of with fuzzy left and right asymmetry amendment, fuzzy transmission, take
The geometric average of left and right.
Additionally, most expecting above-mentioned geometric average from from the viewpoint of fuzzy left and right asymmetry amendment, but for example exist
In the case of such computing corresponding to brighter image is considered in shading value amendment, the benchmark image becomes following formula.
<M(x)>=<Lt(x)>·<Rt(x)>
Now, the final formula defined in step 4 below becomes the form of root of making even.
4) generation of gain profiles data
Using the smoothing image and the benchmark image of 3D rendering amendment of left and right, generate for correcting left view difference image
Gain profiles data gL(x, y) and gain profiles data g for correcting right anaglyphR(x, y).Thus, image structure information
Offset, generate the composition reciprocal of the composition and shading value acted on as fuzzy asymmetry amendment.Here, without remaining
The composition of impact can be brought to parallax.
5) the asymmetric fuzzy revising in the left and right of anaglyph and shading value amendment
Calculated left gain profiles data and right gain profiles data are used respectively, to left view difference image and right disparity map
Asymmetric as each self-correction or so obscures.This also means that while carrying out shading value amendment.By the asymmetric mould of amendment left and right
The left view difference image that paste is obtained is set to Lt ' (x, y), and the asymmetric fuzzy right anaglyph for obtaining of amendment left and right is set to into Rt '
(x, y).
Lt ' (x, y)=gL(x, y) Lt (x, y)
Rt ' (x, y)=gR(x, y) Rt (x, y)
Thus, the revised solid of asymmetry of the fuzzy left and right for generating the characteristic according to parallax pixel is generated
Image, when stereoscopic display is carried out to it, can obtain fuzzy seeming natural stereo-picture.In addition, it is also possible to while
To will not cause based on the natural stereo-picture of the retinal rivalry of shading value.I.e. it is capable to while correct one-eyed stereo
The intrinsic fuzzy left and right asymmetry of shooting and shading value, and generate high-precision stereogram while parallax can not be lost
Picture.
Additionally, shading value is not limited to only relate to the function of the pupil location of optical system and pupil diameter, even if being shot
In the case of the extremely near situation of object and infinity, it is also possible to confirm that its form is different by test.In the present embodiment, due to
Correction is calculated from the local data of image itself, it is possible to realizing also been removed the amendment of the impact.
Embodiments ＞ of ＜ the 2nd
In the present embodiment, the situation for carrying out parallax amount regulation is illustrated.The order of process is substantially as follows.Additionally, with regard to
Under the input of 1) parallax image data, the generation of 4) gain profiles data, it is identical with the 1st embodiment, therefore omit the description.
1) parallax image data input
2) generation of variable smoothing image
3) generation of the benchmark image of 3D rendering amendment
4) generation of gain profiles data
5) parallax amount of anaglyph is adjusted
Hereinafter, illustrate in order.
2) generation of variable smoothing image
Computing is identical with the step of 1 embodiment 2.But, by filtering diameter σ0Value be set to shown in the 1st embodiment
Value to variable between 0.
As the method for two dimension smoothing, in addition to being filtered in real space shown in the 1st embodiment, additionally it is possible to
Changed using multiple resolution.For example, gaussian pyramid can be built.Figure 12 is the figure for illustrating gaussian pyramid.Additionally, with regard to
The situation of gaussian pyramid refer to non-patent literature 1.Here is simply illustrated.In order to prevent folding when generating downscaled images
Repeatedly (aliasing), the smoothing of 5 × 5 degree is carried out, carries out sub-sampling with 1/2 and generate downscaled images Ims.If repeating the mistake
Cheng Ze can obtain the gaussian pyramid being made up of a series of downscaled images.Its series is set to into M levels.As series is raised,
Resolution is transferred to low resolution from high-resolution.
Above-mentioned variable smoothing image, can only to the diminution of minimum resolution on the basis of M levels are freely changed
Image is carried out based on the amplification of linear interpolation until actual resolution.Generally, install in simple eye pupil cutting shooting mode
In the case of the imaging apparatus of 10000000 pixel degree, it is to be analysed that can obtain with the smoothing of maximum disparity same degree
When the conversion of picture degree is up to 5～6 stage.Below it, become the smoothing image in the region that parallax coexists with eradicating efficacy.This
Outward, it is also possible to do not smoothed and down-sampling with same ratio in the horizontal direction and the vertical direction, and entered in varing proportions
Row smoothing and down-sampling.For example, by the resolution of the retention horizontal direction compared with vertical direction, and lengthwise can be generated
Downscaled images.
Non-patent literature 1
P.J.Burt and E.H.Adelson,”The Laplacian pyramid as a compact image
code,”IEEE Trans.Commun.,vol.COM-31,No.4,pp.532-540,1983.
5) parallax amount of anaglyph is adjusted
Computing is identical with the step of 1 embodiment 5.But, remain with step 2 set filtering diameter size into
The parallax amount of direct ratio.Therefore, it is possible to provide parallax amount governor motion.Relational expression is stated as follows.
Parallax amount=constant times × (filtering diameter/σ0)
Constant times are the maximum disparity amounts that the image has.
As described above, in the present embodiment, according to by the data of which level of multiple exploring be used for correct, Neng Gousui
Correction effect to adjust parallax amount.Specifically, if how not smoothed and the high downscaled images of resolution using
The smoothed data of amplification is modified, then remain parallax component, therefore, the improvement of fuzzy asymmetric property is improved.
In this case, because by the mixing of mutual parallax, parallax amount reduces.On the other hand, if using by the low diminution figure of resolution
As the smoothed data for amplifying is modified, then the smoothed data that the high downscaled images of resolution are amplified is repaiied with utilizing
Positive situation is compared, and is reduced relative to the improvement of fuzzy asymmetry little fuzzy (blurred area is little), but parallax into
Partial volume is easily eliminated, therefore, it is possible to suppress the reduction of parallax amount before and after correcting.Additionally, image processing part can be according to by user input
The filtering diameter of setting is adjusting parallax amount.Image processing part can also adjust parallax according to filtering diameter set in advance
Amount.From the multiple parallax amounts by menu presentation one can also be selected to set parallax amount by user.
Embodiments ＞ of ＜ the 3rd
As a example by enumerating the sparse parallax pixel array of bayer-like RGB, specifically illustrate.Figure 11 is denoted as one
The arrangement of real space and the figure of k-space.It is shown with periodically configuring the arrangement figure on the upper hurdle of Figure 11 as primitive lattice
Imaging apparatus example.For the frequency exploring region in its reciprocal lattice space, also by the combination of each color and each parallax
Illustrate.The arrangement is this property of generation parallax only in the subject region for having obscured for capturing simple eye pupil cutting mode
Matter, by the density of parallax pixel sparse configuration is carried out, and residual pixel is assigned as far as possible taking the photograph for the construction of no-parallax pixel
Element.No-parallax pixel and there is parallax pixel using Bayer array as essential structure, be in left parallax pixel and right parallax
R is each equipped with pixel:G:B=1:2:The construction of 1 colored filter.That is, focus on by original signal catch it is parallax free in
Between image exploring power, by parallax modulation its radio-frequency component is overlapped in into left parallax pixel and right parallax pixel, thus obtain
The stereo-picture of high exploring.The order of process is substantially as follows.
1) color parallax multiplex mosaic image data input
2) the global gain balance amendment of color parallax mosaic image
3) generation of interim anaglyph
4) the asymmetric fuzzy revising in the left and right of interim anaglyph (the shading value amendment of interim anaglyph)
(region gain balance amendment)
5) local illumination distribution amendment, no parallax color mosaics image the generation based on left and right
(local gain balance amendment)
6) generation of no parallax benchmark image
7) generation of actual anaglyph
8) to the conversion of output color space
Hereinafter, illustrate in order.
1) color parallax multiplex mosaic image data input
One-board mosaic image after the color of Figure 11 and parallax multiplex is represented with M (x, y).GTG is based on A/
The linear gray level that D is changed and exported.
2) the global gain balance amendment of color parallax mosaic image
Directly using the subject picture that obtains is shot, overall average of image of the pixel value of no-parallax pixel is calculated
ValueThe overall mean value of the image of the pixel value of left parallax pixelThe image of the pixel value of right parallax pixel is overall to put down
AverageIt is different from the 1st embodiment, there are three kinds of signal levels.First, as datum mark between left and right, so that signal is electric
Heibei provincial opera is combined into the mode of mean value and carries out gain-boosted op amp.Now the acquisition methods of datum mark can consider that arithmetic average and geometry are flat
Both.Then, geometric average is taken between the signal level and the signal level of no-parallax pixel after left and right is average, so that
Signal level blending carries out gain-boosted op amp for the mode of the mean value.To RGB, each color component carries out the process.By respective institute
Corresponding mean value is rewritten by following formula.
For ease of explanation in mosaic image M (x, y),
The signal face of the no-parallax pixel of R component is expressed as into RN_mosaic(x,y)、
The signal face of the left parallax pixel of R component is expressed as into RLt_mosaic(x,y)、
The signal face of the right parallax pixel of R component is expressed as into RRt_mosaic(x,y)、
The signal face of the left parallax pixel of G component is expressed as into GN_mosaic(x,y)、
The signal face of the no-parallax pixel of G component is expressed as into GLt_mosaic(x,y)、
The signal face of the right parallax pixel of G component is expressed as into GRt_mosaic(x,y)、
The signal face of the no-parallax pixel of B component is expressed as into BN_mosaic(x,y)、
The signal face of the left parallax pixel of B component is expressed as into BLt_mosaic(x,y)、
The signal face of the right parallax pixel of B component is expressed as into BRt_mosaic(x,y)。
A) in the case of taking arithmetic average between left and right
Mean value
Yield value to no-parallax pixel
Yield value to left parallax pixel
Yield value to right parallax pixel
Global gain amendment to no-parallax pixel
Global gain amendment to left parallax pixel
Global gain amendment to right parallax pixel
B) in the case of taking geometric average between left and right
Mean value
Yield value to no-parallax pixel
Yield value to left parallax pixel
Yield value to right parallax pixel
Global gain amendment to no-parallax pixel
Global gain amendment to left parallax pixel
Global gain amendment to right parallax pixel
When whole no-parallax pixels have the mask of open by the way of arithmetic average type.In whole no parallax pictures
During mask of the element with half opening by the way of geometric average type.Therefore, in the present embodiment using arithmetic average type.This
Sample, by no-parallax pixel with a gain coefficient amendment, by left parallax pixel with a gain coefficient amendment, by right parallax pixel
The mosaic image obtained using a gain coefficient amendment y) is exported as M'(x.
3) generation of interim anaglyph
Generate the interim left view difference image and interim right anaglyph of the low resolution ratio of spatial frequency resolution.Carry out
Only have accumulated the simple average interpolation in the G colors face of left parallax pixel.It is corresponding to ratio of distances constant using the pixel value of close presence
Carry out linear interpolation.Similarly, only have accumulated the simple average interpolation in the G colors face of right parallax pixel.Similarly, enter
Row only have accumulated the simple average interpolation in the G colors face of no-parallax pixel.Same process is carried out respectively to R, G, B.That is, according to
RLt_mosaic(x, y) generates RLt(x, y), according to RRt_mosaic(x, y) generates RRt(x, y), according to RN_mosaic(x, y) generates RN(x,
Y), according to GLt_mosaic(x, y) generates GLt(x, y), according to GRt_mosaic(x, y) generates GRt(x, y), according to GN_mosaic(x, y) is raw
Into GN(x, y), according to BLt_mosaic(x, y) generates BLt(x, y), according to BRt_mosaic(x, y) generates GRt(x, y), according to BN_mosaic
(x, y) generates GN(x,y)。
Interim R component ignores difference image：RN(x,y)
Interim G component ignores difference image：GN(x,y)
Interim B component ignores difference image：BN(x,y)
The left view difference image of interim R component：RLt(x,y)
The left view difference image of interim G component：GLt(x,y)
The left view difference image of interim B component：BLt(x,y)
The right anaglyph of interim R component：RRt(x,y)
The right anaglyph of interim G component：GRt(x,y)
The right anaglyph of interim B component：BRt(x,y)
In addition it is also possible to interim ignore difference image R generatingN(x,y)、GN(x,y)、BNWhen (x, y), signal face is imported
Interior direction determining and accurately carry out.
4) the asymmetric fuzzy revising in the left and right of interim anaglyph and shading value amendment
(region gain balance amendment)
The shading value amendment of the 1st embodiment or the 2nd embodiment is carried out to each faces of RGB.The local gain of aftermentioned step 4
Balance amendment in, be completely eliminated to generate 2D images parallax and while also perform shading value amendment, in contrast, opposite
The asymmetric fuzzy revising of shading value amendment as residual parallax and left and right is carried out into the anaglyph required for 3D rendering.Cause
This, carries out the integration of the average illumination level in local area.
Additionally, not to the stereoscopic display of aftermentioned step 7 with anaglyph carry out and interim anaglyph carried out
The reason for gain balance amendment is GTG may to be produced in the region that the gain by gain integration is lifted entirely white but logical
The parallax modulation treatment of the step of after crossing 7 is taken into the information for ignoring difference image of high exploring, can be by the complete white impact of GTG
All effectively make up.In addition, carrying out there is also reason before following step 5.That is, the no parallax Marseille of 2D generations
Gram image is generated in the interim anaglyph information after being taken into the asymmetric fuzzy revising in the left and right.Therefore, because scheming to 2D
The fuzzy Bayer array image of two-wire is alleviated as also generating, so being not only stereoscopically displaying images, can also be produced and 2D is schemed
The effect that the fuzzy asymmetry of picture is modified.
5) local illumination distribution amendment, no parallax color mosaics image the generation based on left and right
(local gain balance amendment)
Next under the global gain amendment identical form of thinking for carrying out in step 1, by carrying out pixel list
The local gain amendment of position, makes first the illumination blending of the right parallax pixel in the left parallax pixel and picture in picture.It is logical
Cross operation elimination parallax between left and right.On this basis in the image pickup signal for taking the average signal face in left and right and no-parallax pixel
Concoct further illumination between face.So, generates in the new Bayer face after gain is integrated that take in whole pixels.It is equal to and puts down
Average is replaced, therefore, it is possible to generate the Bayer face that parallax is eliminated.It is denoted by MN(x,y)。
In this case, in the establishing method of the datum mark as each pixel and neat desired value, for eliminating left and right
Between parallax method, there is also select arithmetic average method and select geometric average method both approaches.In whole
When no-parallax pixel has the mask area of open, in order that eliminating the fuzzy amplitude of the subject picture of parallax between left and right
It is consistent with the fuzzy amplitude of open and need select arithmetic average type.On the other hand, have in whole no-parallax pixels half-open
During the mask area of mouth, in order that eliminating the fuzzy amplitude of the subject picture of parallax and the fuzzy amplitude of half opening between left and right
Unanimously need to select geometric average type.
And, with regard to being averaged between the image pickup signal face of the signal face and no-parallax pixel that eliminate parallax between left and right
Operation, be the subject picture with identical fuzzy amplitude because both are already concocted, thus need preserve its fuzzy width
Degree.Therefore, geometric average now must commonly be taken.It is exemplified below its concrete formula.
A) in the case of taking arithmetic mean between left and right
The mean value of each pixel
Yield value to each pixel of no-parallax pixel
Yield value to each pixel of left parallax pixel
Yield value to each pixel of right parallax pixel
Local gain amendment to each pixel of no-parallax pixel
Local gain amendment to each pixel of left parallax pixel
Local gain amendment to each pixel of right parallax pixel
B) in the case of taking geometric average between left and right
The mean value of each pixel
Yield value to each pixel of no-parallax pixel
Yield value to each pixel of left parallax pixel
Yield value to each pixel of right parallax pixel
Local gain amendment to each pixel of no-parallax pixel
Local gain amendment to each pixel of left parallax pixel
Local gain amendment to each pixel of right parallax pixel
Like this by the mean value of the image of left view point and the image of right viewpoint further with parallax free referenced viewpoints
Image average obtained from pixel value as new no-parallax pixel value, by the data rewriting in Bayer face, output is ignored
The image M in poor Bayer faceN(x,y)。
6) generation of no parallax benchmark image
Using known Bayer interpolation technique (inverse mosaic processing).For example, refer to the same inventor's of the application
USP8,259,213.The image can directly be printed out as the 2D images of common high exploring and used.
7) generation of actual anaglyph
Using the coloured image R of the low interim left view difference of the exploring power generated in step 3Lt(x,y)、GLt(x,y)、BLt
The high parallax free coloured image R of the exploring power that generates as intermediate treatment in (x, y) and step 5N(x,y)、GN(x,y)、BN
(x, y), generate reality output exploring power it is high left view difference coloured image R'Lt(x,y)、G'Lt(x,y)、B'Lt(x,y).Together
Sample ground, using the coloured image R of the low interim right parallax of the exploring power generated in step 3Rt(x,y)、GRt(x,y)、BRt(x,
Y) the high parallax free coloured image R of the exploring power that generates as intermediate treatment and in step 5N(x,y)、GN(x,y)、BN(x,
Y), the coloured image R' of the high right parallax of the exploring power of generation reality outputRt(x,y)、G'Rt(x,y)、B'Rt(x,y)。
As parallax modulation mode, it is considered to take on the basis of arithmetic average put method and take on the basis of geometric average put
Method both approaches.No matter which method can obtain parallax modulation effect, but in the no-parallax pixel of imaging apparatus
Aperture mask is adopted on the basis of arithmetic average in the way of point when being open, the aperture mask of no-parallax pixel with have parallax
Pixel is adopted in the way of being put on the basis of geometric average when being identical half opening.Therefore, adopt in the present embodiment with arithmetic
The mode of average out to datum mark.
A) the parallax modulation put on the basis of arithmetic average
Left view difference modulation
Right parallax modulation
B) the parallax modulation put on the basis of geometric average
Left view difference modulation
Right parallax modulation
Above formula is rewritten, then
Left view difference modulation
Right parallax modulation
8) to the conversion of output color space
The parallax free middle color image R of the high exploring to so obtainingN(x,y)、GN(x,y)、BN(x, y) and high analysis
The coloured image R of the left view difference of pictureLt(x,y)、GLt(x,y)、BLtThe coloured image R of the right parallax of (x, y), high exploringRt(x,y)、
GRt(x,y)、BRt(x, y), respectively the camera RGB from the dichroism of sensor carry out color to the sRGB color spaces of standard
Matrixing and gamma transformation, and export as the image of output color space.
The process of described above is only pulled out between asymmetry amendment and gain-boosted op amp and parallax amount with regard to obscuring
Simultaneously summarized again the part of relation.For convenience of description, only discuss and geometric average is taken as the datum mark of gain-boosted op amp
Situation.
In local gain balance amendment, in order to generate 2D images, while carrying out the increasing between Lt images and Rt images
Benefit balance is integrated and parallax is eliminated, and has imported 2D image intensities amendments.As its core formula to left view difference image and
Right anaglyph carries out respectively following gain-boosted op amp.
On the other hand, from 2D images by parallax modulate generate 3D rendering process be, as its core formula to nothing
Anaglyph carries out following gain-boosted op amp.
That is, the formula of local gain balance amendment and the formula of parallax modulation are reciprocal relation, and are disappeared along with parallax
Remove.
And, the process of fuzzy left and right asymmetry amendment and shading value amendment is carried out to 3D rendering, i.e. with regard to area
Domain gain balance is modified to, and following gain-boosted op amp is carried out to anaglyph as the formula of its core.
It to play and do not lose parallax information by taking the local average in bigger region and not lose the information of fuzzy behaviour
The effect extracted with brightness information.
Three above-mentioned processes, can say are closed correspondingly with parallax using obscuring in one-eyed stereo shooting
The computing of the derived intrinsic property of system.
In the explanation with regard to the 3rd embodiment more than, to eliminating the signal face and no parallax picture of parallax between left and right
The operation being averaged between the image pickup signal face of element, fuzzy amplitude is being set to have used geometric average under common purpose.
In the case of calculating the geometric average between the mean value of the pixel value of no-parallax pixel and the parallax pixel of left and right, to the pixel
Distribution of the weighted sum of value to the weighting of the mean value is impartial.On the other hand, the quantity of parallax pixel is less than no parallax picture
The quantity of element.And, the exploring power of anaglyph is less than the exploring power for ignoring difference image.As described above, for example as no parallax
The R of imageN、BNNyquist extreme property be by kx=[± π/(2a), ± π/(2a)], ky=[± π/(2a), ± π/
(2a) region] being combined into, in contrast, as the G of anaglyphLt、GRtNyquist extreme property be by kx=
The region that [± π/(4a), ± π/(4a)], ky=[± π/(4a), ± π/(4a)] are combined into.Therefore, if making for no parallax
The pixel value of pixel and for left and right parallax pixel mean value weighting distribution it is impartial, then the exploring power of the image for obtaining
It is overall due to the impact of the exploring power of anaglyph to reduce.Thus, it is desirable to try as close possible to the exploring for ignoring difference image
Power.Accordingly, it is considered to the density ratio of no-parallax pixel in the pixel arrangement on imaging apparatus and parallax pixel and take geometric average
Preferably.Specifically, used in the 3rd embodiment no-parallax pixel (N), left parallax pixel (Lt) and right parallax pixel (Rt) it
Than for N：Lt：Rt=6：1：1, i.e. N：(Lt+Rt)=3：1, therefore, the weighting of 3/4 power is given to ignoring difference image, to regarding
Difference image gives the weighting of 1/4 power, becomes and payes attention to the high distribution for ignoring difference image of density.
As described above, in the method for eliminating parallax between left and right, there is the method and selection geometry for selecting arithmetic average
Average method both.In the case where whole no-parallax pixels have the mask area of open, in order that between left and right
The fuzzy amplitude that eliminates the subject picture of parallax is consistent with the fuzzy amplitude of open and select arithmetic average type preferably.With
Under the situation that have selected arithmetic average type a) is shown.
A) in the case of taking arithmetic average between left and right
The mean value of each pixel
Yield value to each pixel of no-parallax pixel
Yield value to each pixel of left parallax pixel
Yield value to each pixel of right parallax pixel
On the other hand, when whole no-parallax pixels have the mask area of half opening, can be with order that disappearing between left and right
Geometric average type is selected except the fuzzy amplitude of the subject picture of parallax is consistent with the fuzzy amplitude of half opening.Following b)
The situation that have selected geometric average type is shown.
B) in the case of taking geometric average between left and right
The mean value of each pixel
Yield value to each pixel of no-parallax pixel
Yield value to each pixel of left parallax pixel
Yield value to each pixel of right parallax pixel
In addition, carry out parallax modulate when, it is also possible to take by each parallax pixel in the pixel arrangement on imaging apparatus that
The geometric average that the density ratio of the RGB between this is taken into account.That is, it is R between left parallax pixel：G：B=1：2：1,
Also it is R between right parallax pixel：G：B=1：2：1, therefore give adding for 1/4 power to the parallax modulation based on R component
Power, to the parallax modulation based on G component the weighting of 1/2 power is given, and to the parallax modulation based on B component 1/4 power is given
Weighting, using the distribution paid attention to based on the parallax modulation of the high G component of density.Following a) illustrates the point on the basis of arithmetic average
Parallax modulation.
A) the parallax modulation put on the basis of arithmetic average
Left view difference modulation
Right parallax modulation
Following b) the parallax modulation put on the basis of geometric average is shown.
B) the parallax modulation put on the basis of geometric average
Left view difference modulation
Right parallax modulation
Above formula is rewritten, then
Left view difference modulation
Right parallax modulation
＜ supplements ＞
For the arrangement of the 3rd embodiment, understand the 2nd embodiment if the relation of the 1st embodiment and the 2nd embodiment is noted
Parallax control Mechanism Combination, therefore omit the description.
Also the equipment such as personal computer can be made as undertaking image data generating section 230, smoothed data generating unit
231st, destination image data generating unit 232, the image processing apparatus of the function in gain-boosted op amp portion 233 and function.At image
Reason device can be taken into parallax image data by other devices from camera etc..In this case, image data generating section 230 undertakes work
For the function of image data acquisition part.Additionally, in the case where image data generating section 230 oneself generates parallax image data,
Also can say it is to generate by self and obtain parallax image data.Image processing apparatus are not limited to personal computer, can adopt
Use various modes.For example, TV, mobile phone, game machine etc. are with display part or the equal energy of equipment being connected with display part
Enough it is image processing apparatus.Additionally, in the above description, there is the situation that image refers to view data, there is also image is
Refer to that visual subject is as the situation of itself in accordance with form expansion.
[the 2nd embodiment]
Figure 13 is the figure of the structure of the digital camera 10 for illustrating embodiments of the present invention.There is digital camera 50 conduct to take the photograph
The photographic lens 60 of shadow optical system, will guide along the subject light beam of the incidence of optical axis 61 to imaging apparatus 400.Photography is saturating
Mirror 60 can be can be relative to the changeable type lens of the assembly and disassembly of digital camera 50.Digital camera 50 has imaging apparatus 400, control
Portion processed 501, A/D change-over circuits 502, memory 503, drive division 504, image processing part 505, storage card IF507, operating portion
508th, display part 509 and LCD drive circuits 510.
Additionally, as illustrated, the direction parallel with the optical axis 61 towards imaging apparatus 400 is set to into+Z-direction, will be
It is set to+X-direction towards the direction in paper in plane orthogonal to Z-axis ,+Y-axis side will be set to towards the direction above paper
To.It is that X-axis is horizontal direction with the relation found a view to photograph, Y-axis is vertical direction.
Photographic lens 60 is made up of multiple optical lens groups, makes the subject light beam from scene near its focus face
Imaging.Additionally, in fig. 13 for convenience of description, taken the photograph to represent as representative with being configured in the imaginary 1 piece of lens near pupil
Shadow lens 60.
Imaging apparatus 400 is configured near the focus face of photographic lens 60.Imaging apparatus 400 is to be configured with two dimensions
Such as CCD, cmos sensor etc. imageing sensor of multiple photo-electric conversion elements.Imaging apparatus 400 is by the timing control of drive division 504
System, the subject picture being imaged on sensitive surface is converted into picture signal and is exported to A/D change-over circuits 502.To A/D conversions
Picture signal of the picture signal of the output of circuit 502 comprising left view point and right viewpoint.
The picture signal that imaging apparatus 400 is exported is converted into data image signal and to memory by A/D change-over circuits 502
503 outputs.Memory 503 is implemented various image procossings as working space by image processing part 505, so as to generate picture number
According to.Especially, image processing part 505 has image data generating section 530, One Dimensional Projection data generating section 531, one-dimensional smoothing
Data generating section 532, one-dimensional reference data generating unit 533 and shading value correction portion 534.
Image data generating section 530 is generated corresponding with left view point using left view point and the data image signal of right viewpoint
Left parallax image data and right parallax image data corresponding with right viewpoint.With regard to left parallax image data and right anaglyph number
According to the details of generation will be described later.
One Dimensional Projection data generating section 531 left parallax image data and right parallax image data are added respectively with by a left side
The pixel value of the multiple pixels arranged on the orthogonal direction of viewpoint and rectilinear direction, the i.e. parallax directions of right viewpoint connection.Thus,
Generate the One Dimensional Projection data and right viewpoint corresponding with right parallax image data of left view point corresponding with left parallax image data
One Dimensional Projection data.
The One Dimensional Projection number of one-dimensional smoothed data generating unit 532 One Dimensional Projection data respectively to left view point and right viewpoint
According to this amplitude of samples more than maximum disparity amount calculates rolling average, thus generates the one-dimensional smoothed data of left view point and the right side regards
The one-dimensional smoothed data of point.The generation of the one-dimensional smoothed data of left view point and the one-dimensional smoothed data of right viewpoint it is detailed
Situation will be described later.
The one-dimensional smoothing number of one-dimensional smoothed data and right viewpoint of the one-dimensional reference data generating unit 533 based on left view point
According to generating one-dimensional reference data.Specifically, the one-dimensional smoothed data using left view point and the one-dimensional smoothing of right viewpoint
The geometric average or arithmetic average of the respective pixel value corresponding to data is generating one-dimensional reference data.One-dimensional reference data
The details of generation will be described later.
Shading value correction portion 534 is repaiied using one-dimensional reference data relative to the ratio of the one-dimensional smoothed data of left view point
Just left parallax image data.In addition, being corrected relative to the ratio of the one-dimensional smoothed data of right viewpoint using one-dimensional reference data
Right parallax image data.Thereby, it is possible to obtain the revised left parallax image data of shading value and right parallax image data.
Image processing part 505 also undertakes according to selected picture format to adjust the image of view data etc. in addition
The general function of process.The view data for being generated is converted into showing signal by LCD drive circuits 510, and is displayed in aobvious
Show in portion 509.In addition, record is in the storage card 520 of storage card IF507 is installed on.
A series of photographic process accepts the operation of user and to the output function of control unit 501 letter by operating portion 508
Number and start.The various actions such as AF, the AE with photographic process are controlled by control unit 501 and performed.
In the digital camera that simple eye pupil cutting images mode, in addition to fuzzy asymmetry, also with left and right
Anaglyph between produce the problem of different shading values.Additionally, in this manual, also the shading value of anaglyph is denoted as
Parallax shading value.In WO2012/036019 publications, the skill of the shading value to correct anaglyph is described using correction chart
Art.Specifically, for the purpose of the shading value for correcting anaglyph, uniform surface is shot in advance, for all optical strips
The dimensional table for amendment is made and stored to part in advance.Then, in the case of amendment shading value, the optical condition being consistent is read
Correction chart carrying out shading value amendment.The state-contingent of shading value is in the pupil location (focal length) and pupil of optical system
Diameter (f-number) and change, therefore for shading value amendment is carried out using correction chart, need advance for these full terms
Setting data.In addition, in the embodiment of WO2012/036019 publications, it is assumed that in left view difference image and right anaglyph
The middle shading value characteristic that left and right reversion is presented, by flexibly reversally reducing using dimensional table memory.
The mistake of the impact of the shading value of the stereoscopically displaying images that mode is generated is imaged with simple eye pupil cutting in probe
Cheng Zhong, has distinguished following facts.
1) shading value of anaglyph has a pupil diameter less (f-number is bigger) and the shading value difference of left and right is bigger
Tendency.
2) parallax shading value because of the impact of the foozle of parallax pixel etc. generally in left view difference image and right disparity map
It is asymmetric as between.
3) when will not correct the image stereoscopic display of parallax shading value and observe, can produce near the end of image
The caused retinal rivalry because of the luminance difference of right anaglyph and left view difference image, can bring strong incongruity, for
The reason for long-time observation stereo-picture becomes generation fatigue.
The technology described in WO2012/039346 is had studied in detail, its result is that the 1st of WO2012/039346 implements
In technology described in mode, distinguish actually with shading value amendment simultaneously, left view difference image (A images) and right anaglyph
(B images) loses respectively parallax information, even if they are shown as into stereo-picture also can only become plane picture.That is, if not having
There is the relation for considering shading value amendment and parallax residual quantity, then may lose stereoeffect itself.In the 2nd embodiment, if
Method all smooths left view difference image (A images), right anaglyph (B images) and composograph (C images), but flat due to this
The definition of cunningization is ambiguous and indefinite, it is possible that causing forfeiture parallax identical with the 1st embodiment and cannot show stereogram
Picture.Therefore, in the 3rd embodiment, describe only carries out parallax shading value amendment in display plane image, in stereo-picture
It is not modified during display pattern.That is, in the impact that parallax shading value is corrected using the commonness photograph image information of itself
In the case of, parallax information may be lost and stereo-picture itself cannot be generated.
On the other hand, the spy in the method for WO2012/036019, due to only correcting the parallax shading value for precomputing
Property, so while being not concerned about that parallax information can be lost, but need to keep in advance with regard to the lensed all zoom positions of institute and light
The data of circle value.Therefore, it is extremely difficult to install in reality, even if exist to complete to install the labour's time that also can be expended in advance
Problem.
In the present embodiment, illustrate in view of these situations, using commonness photograph image, the information of itself presses each image
Parallax shading value is simply corrected, and does not lose the shading value amendment skill for stereoscopic display of function as stereo-picture
Art.Especially, in the 1st to the 3rd embodiment, illustrate that the image for already generating stereo-picture to carrying out one-eyed stereo shooting is repaiied
Just because of the technology of shading value caused by parallax pixel.In the 4th embodiment, in order to express the shading value amendment of anaglyph and regard
Relation between difference elimination, being obtained with parallax while shooting color by veneer sensor in illustrating to one-eyed stereo camera head
Image and the technology of jointly restoring both color and parallax.
＜ fuzzy isotropization and parallax eliminates ＞
Describe following facts in detail in Japanese Patent Application 2012-179044 of the applicant：For in parallax pixel
What is produced between horizontal direction and vertical direction is asymmetric fuzzy, by apply to smooth wave filter in the horizontal direction and by mould
Paste isotropization, and become and seem naturally fuzzy, and with the isotropization progress and parallax reduces, finally
After completely isotropic, parallax is completely eliminated.That is, as the fuzzy of horizontal direction is connect by the smoothing of horizontal direction
The fuzzy amplitude of nearly vertical direction, parallax amount reduces, and final parallax is eliminated.In other words, with connecing from the fuzzy amplitude of half opening
The fuzzy amplitude of nearly open, the parallax amount at initial stage when shooting is changed to parallax zero.Therefore, the smoothing journey of horizontal direction
Degree can act as parallax amount control device.Efficient, flexible is used can make parallax be zero by the smoothing of the horizontal direction
It is true.In more detail, the smoothing filter of the fixation with filtered amplitude more than maximum disparity amount is applied in the horizontal direction
Ripple device.Thereby, it is possible to be eliminated the anaglyph of parallax all the time.
Embodiments ＞ of ＜ the 1st
In the present embodiment, situation about being processed the stereo-picture of black and white is illustrated.In the colour formed by the colors of RGB tri-
In the case of image, as long as carrying out to assorted face and the process of black white image identical, therefore omit the description.The order of process
It is substantially as follows.
1) parallax image data input
2) generation of One Dimensional Projection data
3) generation of one-dimensional smoothing distributed data
4) generation of one-dimensional benchmark distributed data
5) generation of one-dimensional gain profiles data
6) the shading value amendment of anaglyph
Hereinafter, illustrate in order.
1) parallax image data input
It is input into and left view difference image Lt (x, y) and right anaglyph Rt (x, y) for obtaining is shot to common subject picture.This
When, each anaglyph is the linear gray level data for not carrying out gamma-corrected.In addition, in the case where coloured image is processed,
The RGB data being contemplated to be under the sensor light spectral sensitivity original state for not carrying out color treatments.
2) generation of One Dimensional Projection data
For the brightness information that only appropriate extraction is contained in anaglyph, the picture number from common subject picture is needed
According to elimination image structure information.In addition, as noted previously, as parallax should not be eliminated because carrying out shading value amendment, so also needing
The parallax information included in anaglyph is set also to lose.In this step, in order to eliminate image structure information in early days, using right
The anaglyph of two dimension vertically all carries out data addition, and the method for carrying out One Dimensional Projection in the horizontal direction.If
The mean value that the data that One Dimensional Projection is obtained are taken as obtaining the additive value of vertical direction divided by data bulk, then obtain noise
The One Dimensional Projection data that are also inhibited of fluctuating component.The situation of the parallax distribution of narration One Dimensional Projection data, subject
As in certain region focusing, certain region has parallax, therefore the One Dimensional Projection data that vertical summation is obtained for fuzzy region to regard
The difference mixed and disorderly distribution weaker than maximum disparity amount is present.
The One Dimensional Projection data of left view difference image and right anaglyph are represented respectively with Lt (x), Rt (x).
Lt (x)=∫ Lt (x, y) dy/ ∫ dy
Rt (x)=∫ Rt (x, y) dy/ ∫ dy
3) generation of one-dimensional smoothing distributed data
Then, by will not the amendment of system of Himdu logic darkness and in the way of eliminating parallax, data for projection and right One Dimensional Projection are tieed up to the first from left
Data apply in the horizontal direction the smoothing wave filter bigger than maximum disparity amount, enter to be about to the place that parallax information is completely eliminated
Reason.Thus, in the One Dimensional Projection data after smoothing, the lenticular characteristic for being contained mainly in parallax pixel is simple eye with its
Produce under the relation between hypothetical pupil in mirror optical system, the shading value composition that anaglyph is intrinsic and One Dimensional Projection
The rough configuration information of image the two information.
Maximum disparity amount between the anaglyph of left and right by the shooting area of imaging apparatus, pel spacing and optical system it
Between relationship specifications, be 5% or so of the horizontal direction order of magnitude for image planes entirety.If horizontal direction is that 2000 pixels are left
It is right then for 100 pixels or so.Apply the smoothing wave filter in corresponding horizontal direction.By filter radius with σ0Represent,
It is set as the order of magnitude of maximum disparity amount.It may be said that this is filtered into local smoothing method, the One Dimensional Projection data after smoothing are with it
The precision of locality keeps shading value composition.
In the case of Gaussian Blur, represented with following formula.In the sense that local average is taken, by the first from left dimension smoothing distribution
Data and right one-dimensional smoothing distributed data are represented respectively by ＜ L (x) ＞, ＜ R (x) ＞.
Gaussian blur filter
The one-dimensional smoothing techniques of horizontal direction
Even if additionally, Gaussian Blur can not be simply used, it is also possible to what is all represented by the positive filter factor of identical
Uniform wave filter is alternative.
4) generation of one-dimensional benchmark distributed data
The first from left according to parallax is eliminated ties up smoothing distributed data and eliminates the right one-dimensional smoothing distribution number of parallax
According to the one-dimensional benchmark distributed data generated for representing the reference grade of shading value amendment.By one-dimensional benchmark distributed data by ＜ M
X () ＞ is represented.The generation method of the datum mark has two kinds.A kind of is the method for the arithmetic average for taking left and right, and another kind is to take
The method of the geometric average of left and right.
In the case of taking arithmetic average
In the case of taking geometric average
5) generation of one-dimensional gain profiles data
Using the one-dimensional smoothing distributed data and one-dimensional benchmark distributed data of left and right, generate for correcting left view difference image
One-dimensional gain profiles data gLt(x) and one-dimensional gain profiles data g for correcting right anaglyphRt(x).Thus, generate
The composition reciprocal of the shading value composition that image structure information is offseted.
There are two kinds of generations for being accordingly, there are two kinds of gain profiles data with the generation method of one-dimensional benchmark distributed data
Method.
In the case of taking arithmetic average
In the case of taking geometric average
6) the shading value amendment of anaglyph
Respectively using calculated the first from left dimension gain profiles data and right one-dimensional gain profiles data, to left view difference image and
Right anaglyph carries out respectively shading value amendment.Use to correct 2-D data by one-dimensional data is all common in each row.
That is, following computings are carried out to each point in y-axis.The revised left view difference image of shading value is set to into Lt ' (x, y), by shading value
Revised right anaglyph is set to Rt ' (x, y).
Lt ' (x, y)=gLt(x) Lt (x, y)
Rt (x, y)=gRt(x) .Rt (x, y)
So, the stereo-picture that the shading value to producing the characteristic because of parallax pixel is corrected is generated.When by its
During stereoscopic display, can obtain that the natural stereo-picture of system of Himdu logic darkness and caused retinal rivalry will not be caused.By by root
The shading value characteristic changed according to optical condition is extracted out from image itself, and can provide simple and reliable shading value amendment side
Method.That is, can be using the commonness photograph image signal message of itself, simply being constituted with do not lose stereoeffect to disparity map
The shading value produced as in is modified, so as to generate natural stereo-picture.
Additionally, generally shading value characteristic is also different in the case of the extremely near situation of subject and infinity.Therefore, it is bright
Darkness is likely to be changed according to the configuration relation of subject picture, but it is only stop down that actually shading value becomes problem
Optical system when.In this case, it is believed that the depth of field is deeper, multiple subjects are in and focus for infinity identical shape
Condition, therefore the shading value amendment of abundant and beneficial stereo-picture also can be provided in the modification method of the present embodiment.In addition,
The anaglyph of the multiple left and right for for associated plurality of or so anaglyph, for example being obtained by continuous shooting shooting, as long as
Shading value amendment is carried out to the anaglyph of at least a certain left and right.The anaglyph of other left and right can not be carried out
Shading value amendment, it is also possible to the direct utilization of gain profiles data used in a certain shading value amendment is carried out into shading value and is repaiied
Just.Thereby, it is possible to reduce the process load with regard to shading value amendment.Furthermore it is also possible to only regard to associated plurality of left and right
It is in difference image, by user input etc. it is set in advance left and right anaglyph carry out shading value amendment.
Figure 14 is the figure of that illustrates shading value amendment.(a) of Figure 14 is to represent the photography shown on display part 209
The figure of image.(b) of Figure 14 is the figure for representing the left view difference image before shading value amendment and right anaglyph.In fig. 14, it is cloudy
The density of hachure represents the degree of brightness.In anaglyph, caused shading value is produced because of the angle interdependence of pixel.Cause
This, such as shown in (b) of Figure 14, left view difference image and right anaglyph are different with the brightness of right part office in left end portion respectively.
In figure, in left view difference image, right end portion is brighter than left end portion, and in contrast, in right anaglyph, left end portion is than right
End part is bright.In addition, parallax shading value is not symmetrical in left view difference image and right anaglyph.That is, in right anaglyph
Left and right two ends brightness difference more than the brightness at the left and right two ends in left view difference image difference.
(c) of Figure 14 is the figure for representing the revised left view difference image of shading value and right anaglyph.As illustrated, passing through
Shading value amendment, the brightness of the left end of left view difference image is roughly the same with the brightness of the left end of right anaglyph.Similarly, left view
The brightness of the right-hand member of difference image is also roughly the same with the brightness of the right-hand member of right anaglyph.Like this, due to left view can be mitigated
The difference of the lightness at the corresponding part of difference image and right anaglyph, it is possible to unplessantness displeasure when mitigating stereos copic viewing.
Embodiments ＞ of ＜ the 2nd
Illustrate in the one-dimensional gain profiles data of calculation procedure 5 using the situation of fitting function.With the 1st embodiment not
It is only that with part, increases the process that the one-dimensional gain profiles data to step 5 are processed further and obtain, therefore only description should
Part.The order of process is identical with the 1st embodiment.
5) generation of one-dimensional gain profiles data
One-dimensional gain profiles data g obtained in 1st embodimentLt(x) and gRtX () take into account exclusion normal image knot
Structure, but input common subject anaglyph with high sensitivity shooting in the case of, may still remaining noise
Fluctuating component.In addition, causing only in the anaglyph of a side comprising extreme bright regions in due to common subject picture
In the case of brightness degree saturation, sometimes cannot appropriate rejection image structural information.In this case, in order that shading value
Amendment is rightly acted on, and is also carried out from one-dimensional gain profiles data forcing the process of rejection image structural information.Specifically, profit
Having with shading value can be with characteristic as Function Fitting gentle all the time.Figure 15 is the figure for illustrating Function Fitting.Figure 15's
Upside illustrates the situation of F11, and downside illustrates the situation of F1.4.In the various figures, transverse axis represents the location of pixels of horizontal direction, the longitudinal axis
Represent signal level.In addition, in the various figures, illustrate the shading value that a line for only taking out homogeneous image is obtained movement situation and
By the situation of its data obtained with 4 Function Fittings.If the one-dimensional gain profiles data to obtaining in the 1st embodiment carry out letter
Number fitting, then obtain the gain profiles data that image structure information is forced to exclude completely.It is considered that due to original light and shade
Degree can be with 4 function representations, so gain profiles also can be with 4 approximations to function.
gLt(x)=aL+bLx+cLx2+dLx3+eLx4
gRt(x)=aR+bRx+cRx2+dRx3+eRx4
Here, constant a, b, c, d, e are respectively fitting coefficient.Coefficient is obtained by minimum multiplication.When being fitted,
Reliably image structure information can be excluded and also excluded the fluctuating of noise.
So, using the one-dimensional gain profiles data approximately obtained by fitting function, shading value is carried out in step 6 and is repaiied
Just.
Embodiments ＞ of ＜ the 3rd
In the 1st embodiment and the 2nd embodiment, by the resolution of input picture with reset condition process, but such as the 2nd enforcement
Example is described, and shading value signal is the composition gently changed in image entirety.Therefore, even if input picture is fallen within for example
The resolution of breviary (thumbnail) image as 360 × 240 pixels also can fully obtain its information.If will input
The reference axis of image is set to (x, y), and by the reference axis of thumbnail image with (x', y') expression, then the minification in thumbnail image is
1/ α times (α ＞ 1) and by sampling starting position with (x0,y0) when representing, following relation is set up.
X=α x'+x0,
Y=α y'+y0。
Thus, on the location of pixels (x', y') of thumbnail image, as long as carrying out identical with the 1st embodiment or the 2nd embodiment
Computing, and on this basis, during the execution of the shading value amendment only in last step 6, using the variable with regard to above-mentioned x
Conversion carries out computing.
The 1st embodiment that can be carried out by thumbnail image as described above and the method for the 2nd embodiment, it may be said that right and wrong
The often simple process of suitable hardware handles.Its reason is that actual anaglyph by streamline between until generated
During stand-by period, with the process that thumbnail image is completed to step 5, can perform while anaglyph is come with pipeline flow
The process of step 6.In addition, the consideration method of the One Dimensional Projection of step 1 can produce the process for being further suitable to hardware.Hard
The extensive smoothing of the filter tap number length of the vertical direction in part in actual resolution needs linear memory, therefore generally
Mostly difficult, in contrast, in the present embodiment, the process under breviary size is easily carried out, and smoothing is limited to horizontal direction.
Embodiments ＞ of ＜ the 4th
Enumerate and specifically illustrated as a example by the sparse parallax pixel array of bayer-like RGB.Here, explanation is using by Figure 11's
The example of the imaging apparatus that the arrangement figure on upper hurdle is configured periodically as primitive lattice.The feature of the arrangement of Figure 11 is as already
Narration is such.The order of process is substantially as follows.
1) color parallax multiplex mosaic image data input
2) the global gain balance amendment of color parallax mosaic image
3) generation of interim anaglyph
4) local illumination distribution amendment, no parallax color mosaics image the generation based on left and right
(local gain balance amendment)
5) generation of no parallax benchmark image
6) generation of actual anaglyph
7) the shading value amendment of anaglyph
8) to the conversion of output color space
Hereinafter, illustrate in order.
1) color parallax multiplex mosaic image data input
One-board mosaic image after the color of Figure 11 and parallax multiplex is represented with M (x, y).GTG is by A/
The linear gray level that D is changed and exported.
2) the global gain balance amendment of color parallax black white image
Directly using the subject picture that obtains is shot, overall average of image of the pixel value of no-parallax pixel is calculated
ValueThe overall mean value of the image of the pixel value of left parallax pixelThe image of the pixel value of right parallax pixel is overall to put down
AverageIt is different from the 1st embodiment, there are three kinds of signal levels.First, as datum mark between left and right, so that signal is electric
Heibei provincial opera is combined into the mode of mean value and carries out gain-boosted op amp.Now the acquisition methods of datum mark can consider that arithmetic average and geometry are flat
Both.Then, geometric average is taken between the signal level and the signal level of no-parallax pixel after left and right is average, so that
Signal level blending carries out gain-boosted op amp for the mode of the mean value.To RGB, each color component carries out the process.By respective institute
Corresponding mean value is rewritten by following formula.
For purposes of illustration only, in mosaic image M (x, y),
The signal face of the no-parallax pixel of R component is expressed as into RN_mosaic(x, y),
The signal face of the left parallax pixel of R component is expressed as into RLt_mosaic(x, y),
The signal face of the right parallax pixel of R component is expressed as into RRt_mosaic(x, y),
The signal face of the left parallax pixel of G component is expressed as into GN_mosaic(x, y),
The signal face of the no-parallax pixel of G component is expressed as into GLt_mosaic(x, y),
The signal face of the right parallax pixel of G component is expressed as into GRt_mosaic(x, y),
The signal face of the no-parallax pixel of B component is expressed as into BN_mosaic(x, y),
The signal face of the left parallax pixel of B component is expressed as into BLt_mosaic(x, y),
The signal face of the right parallax pixel of B component is expressed as into BRt_mosaic(x,y)。
A) in the case of taking arithmetic average between left and right
Mean value
Yield value to no-parallax pixel
Yield value to left parallax pixel
Yield value to right parallax pixel
Global gain amendment to no-parallax pixel
Global gain amendment to left parallax pixel
Global gain amendment to right parallax pixel
B) in the case of taking geometric average between left and right
Mean value
Yield value to no-parallax pixel
Yield value to left parallax pixel
Yield value to right parallax pixel
Global gain amendment to no-parallax pixel
Global gain amendment to left parallax pixel
Global gain amendment to right parallax pixel
When whole no-parallax pixels have the mask of open by the way of arithmetic average type.In whole no parallax pictures
During mask of the element with half opening by the way of geometric average type.Therefore, in the present embodiment using arithmetic average type.This
Sample, by no-parallax pixel with a gain coefficient amendment, by left parallax pixel with a gain coefficient amendment, by right parallax pixel
The mosaic image obtained using a gain coefficient amendment y) is exported as M'(x.
3) generation of interim anaglyph
Generate the interim left view difference image and interim right anaglyph of the low resolution ratio of spatial frequency resolution.Carry out
Only have accumulated the simple average interpolation in the G colors face of left parallax pixel.It is corresponding to ratio of distances constant using the pixel value of close presence
Carry out linear interpolation.Similarly, only have accumulated the simple average interpolation in the G colors face of right parallax pixel.Similarly, enter
Row only have accumulated the simple average interpolation in the G colors face of no-parallax pixel.Identical process is carried out respectively to R, G, B.That is, according to
RLt_mosaic(x, y) generates RLt(x, y), according to RRt_mosaic(x, y) generates RRt(x, y), according to RN_mosaic(x, y) generates RN(x,
Y), according to GLt_mosaic(x, y) generates GLt(x, y), according to GRt_mosaic(x, y) generates GRt(x, y), according to GN_mosaic(x, y) is raw
Into GN(x, y), according to BLt_mosaic(x, y) generates BLt(x, y), according to BRt_mosaic(x, y) generates GRt(x, y), according to BN_mosaic
(x, y) generates GN(x,y)。
Interim R component ignores difference image：RN(x,y)
Interim G component ignores difference image：GN(x,y)
Interim B component ignores difference image：BN(x,y)
The left view difference image of interim R component：RLt(x,y)
The left view difference image of interim G component：GLt(x,y)
The left view difference image of interim B component：BLt(x,y)
The right anaglyph of interim R component：RRt(x,y)
The right anaglyph of interim G component：GRt(x,y)
The right anaglyph of interim B component：BRt(x,y)
In addition it is also possible to interim ignore difference image R generatingN(x,y)、GN(x,y)、BNWhen (x, y), signal face is imported
Interior direction determining and accurately carry out.
4) Illumination Distribution amendment, no parallax color mosaics image the generation based on left and right
(local gain balance amendment)
Next under the global gain amendment identical form of thinking for carrying out in step 1, by carrying out pixel list
The local gain amendment of position, makes first the illumination blending of the right parallax pixel in the left parallax pixel and picture in picture.It is logical
Cross operation elimination parallax between left and right.On this basis in the image pickup signal for taking the average signal face in left and right and no-parallax pixel
Concoct further illumination between face.So, generates in the new Bayer face after gain is integrated that take in whole pixels.It is equal to and puts down
Average is replaced, and can generate the Bayer face that parallax is eliminated.It is denoted by MN(x,y)。
In this case, in the establishing method of the datum mark as each pixel and neat desired value, for eliminating left and right
Between parallax method, there is also select arithmetic average method and select geometric average method both approaches.In whole
When no-parallax pixel has the mask area of open, in order that eliminating the fuzzy amplitude of the subject picture of parallax between left and right
It is consistent with the fuzzy amplitude of open and need select arithmetic average type.On the other hand, have in whole no-parallax pixels half-open
During the mask area of mouth, in order that eliminating the fuzzy amplitude of the subject picture of parallax and the fuzzy amplitude of half opening between left and right
Unanimously need to select geometric average type.
And, with regard to being averaged between the image pickup signal face of the signal face and no-parallax pixel that eliminate parallax between left and right
Operation, be the subject picture with identical fuzzy amplitude because both are already concocted, thus need preserve its fuzzy width
Degree.Therefore, geometric average now must commonly be taken.It is exemplified below its concrete formula.
A) in the case of taking arithmetic mean between left and right
The mean value of each pixel
Yield value to each pixel of no-parallax pixel
Yield value to each pixel of left parallax pixel
Yield value to each pixel of right parallax pixel
Local gain amendment to each pixel of no-parallax pixel
Local gain amendment to each pixel of left parallax pixel
Local gain amendment to each pixel of right parallax pixel
B) in the case of taking geometric average between left and right
The mean value of each pixel
Yield value to each pixel of no-parallax pixel
Yield value to each pixel of left parallax pixel
Yield value to each pixel of right parallax pixel
Local gain amendment to each pixel of no-parallax pixel
Local gain amendment to each pixel of left parallax pixel
Local gain amendment to each pixel of right parallax pixel
Like this by the mean value of the image of left view point and the image of right viewpoint further with parallax free referenced viewpoints
Image average obtained from pixel value as new no-parallax pixel value, by the data rewriting in Bayer face, output is ignored
The image M in poor Bayer faceN(x,y)。
5) generation of no parallax benchmark image
Using known Bayer interpolation technique (inverse mosaic processing).For example, refer to the same inventor's of the application
USP8,259,213.The no parallax benchmark image for so generating has carried out shading value by the local gain amendment of step 4
Amendment, therefore generate the 2D images that the impact of the shading value for producing in the left and right region of anaglyph is eliminated.The image can
Directly print out as the 2D images of common high exploring and use.
6) generation of actual anaglyph
Using the coloured image R of the low interim left view difference of the exploring power generated in step 3Lt(x,y)、GLt(x,y)、BLt
The high parallax free coloured image R of the exploring power that generates as intermediate treatment in (x, y) and step 5N(x,y)、GN(x,y)、BN
(x, y), generate reality output exploring power it is high left view difference coloured image R'Lt(x,y)、G'Lt(x,y)、B'Lt(x,y).Together
Sample ground, using the coloured image R of the low interim right parallax of the exploring power generated in step 3Rt(x,y)、GRt(x,y)、BRt(x,
Y) the high parallax free coloured image R of the exploring power that generates as intermediate treatment and in step 5N(x,y)、GN(x,y)、BN(x,
Y), the coloured image R' of the high right parallax of the exploring power of generation reality outputRt(x,y)、G'Rt(x,y)、B'Rt(x,y)。
As parallax modulation mode, it is considered to take on the basis of arithmetic average put method and take on the basis of geometric average put
Method both approaches.No matter which method can obtain parallax modulation effect, but in the no-parallax pixel of imaging apparatus
Aperture mask be open when select on the basis of arithmetic average put in the way of, the aperture mask of no-parallax pixel with have parallax
Pixel is adopted in the way of being put on the basis of geometric average when being identical half opening.Therefore, adopt in the present embodiment with arithmetic
The mode of average out to datum mark.
A) the parallax modulation put on the basis of arithmetic average
Left view difference modulation
Right parallax modulation
B) the parallax modulation put on the basis of geometric average
Left view difference modulation
Right parallax modulation
Above formula is rewritten, then
Left view difference modulation
Right parallax modulation
7) the shading value amendment of anaglyph
Because the anaglyph of the low exploring before parallax modulation does not carry out shading value amendment, even if so by local
Gain balance amendment is analysed from the impact that difference image eliminates shading value of ignoring of high exploring in the height modulated by parallax and generated
In the anaglyph (i.e. step 6 is in the anaglyph obtained) of picture, shading value also can be again produced.Therefore, each faces of RGB are carried out
The shading value amendment of the 1st embodiment, the 2nd embodiment or the 3rd embodiment.
8) to the conversion of output color space
The parallax free middle color image R of the high exploring to so obtainingN(x,y)、GN(x,y)、BN(x, y) and high analysis
The coloured image R of the left view difference of pictureLt(x,y)、GLt(x,y)、BLtThe coloured image R of the right parallax of (x, y), high exploringRt(x,y)、
GRt(x,y)、BRt(x, y), respectively the camera RGB from the dichroism of sensor carry out color to the sRGB color spaces of standard
Matrixing and gamma transformation, and export as the image of output color space.
The process of described above is only pulled out by the part with regard to the relation between gain-boosted op amp and parallax amount and carried out again
Summarize.In order to make it easy to understand, only discussing the situation that geometric average is taken as the datum mark of gain-boosted op amp.
Local gain balance amendment in, to generate 2D images for the purpose of, while carrying out between Lt images and Rt images
Gain balance integration and parallax eliminate, and imported 2D image intensities amendments.It is poor to left view as the formula of its core
Image and right anaglyph carry out respectively the formula of following gain-boosted op amp.
On the other hand it is to nothing as the formula of its core during generating 3D rendering by parallax modulation from 2D images
Anaglyph carries out the formula of following gain-boosted op amp.
That is, the formula of local gain balance amendment and the formula of parallax modulation are reciprocal relation, and are disappeared along with parallax
Remove.
And, it is that anaglyph is entered as the formula of its core during shading value amendment is carried out to 3D rendering
The formula of the following gain-boosted op amp of row.
Above-mentioned formula is played not to be lost parallax information by taking local average and does not carry out with not losing brightness information
The effect of extraction.
Three above-mentioned processes, can say are closed correspondingly with parallax using obscuring in one-eyed stereo shooting
The computing of the derived intrinsic property of system.
In the explanation with regard to the 4th embodiment more than, to eliminating the signal face and no parallax picture of parallax between left and right
The operation being averaged between the image pickup signal face of element, fuzzy amplitude is being set to have used geometric average under common purpose.
In the case of calculating the geometric average between the mean value of the pixel value of no-parallax pixel and the parallax pixel of left and right, to the pixel
Distribution of the weighted sum of value to the weighting of the mean value is impartial.On the other hand, the quantity of parallax pixel is less than no parallax picture
The quantity of element.And, the exploring power of anaglyph is less than the exploring power for ignoring difference image.As described above, for example as no parallax
The R of imageN、BNNyquist extreme property be by kx=[± π/(2a), ± π/(2a)], ky=[± π/(2a), ± π/
(2a) region] being combined into, in contrast, as the G of anaglyphLt、GRtNyquist extreme property be by kx=
The region that [± π/(4a), ± π/(4a)], ky=[± π/(4a), ± π/(4a)] are combined into.Therefore, if making for no parallax
The pixel value of pixel and for left and right parallax pixel mean value weighting distribution it is impartial, then the exploring power of the image for obtaining
It is overall due to the impact of the exploring power of anaglyph to reduce.Thus, it is desirable to try as close possible to the exploring for ignoring difference image
Power.Accordingly, it is considered to the density ratio of no-parallax pixel in the pixel arrangement on imaging apparatus and parallax pixel and take geometric average
Preferably.Specifically, used in the 4th embodiment no-parallax pixel (N), left parallax pixel (Lt) and right parallax pixel (Rt) it
Than for N：Lt：Rt=6：1：1, i.e. N：(Lt+Rt)=3：1, therefore, the weighting of 3/4 power is given to ignoring difference image, to regarding
Difference image gives the weighting of 1/4 power, becomes and payes attention to the high distribution for ignoring difference image of density.
As described above, in the method for eliminating parallax between left and right, there is the method and selection geometry for selecting arithmetic average
Average method both.In the case where whole no-parallax pixels have the mask area of open, in order that between left and right
The fuzzy amplitude that eliminates the subject picture of parallax is consistent with the fuzzy amplitude of open and select arithmetic average type preferably.With
Under the situation that have selected arithmetic average type a) is shown.
A) in the case of taking arithmetic average between left and right
The mean value of each pixel
Yield value to each pixel of no-parallax pixel
Yield value to each pixel of left parallax pixel
Yield value to each pixel of right parallax pixel
On the other hand, when whole no-parallax pixels have the mask area of half opening, in order that eliminate regarding between left and right
The fuzzy amplitude of poor subject picture is consistent with the fuzzy amplitude of half opening and select geometric average type preferably.Following b) show
Go out have selected the situation of geometric average type.
B) in the case of taking geometric average between left and right
The mean value of each pixel
Yield value to each pixel of no-parallax pixel
Yield value to each pixel of left parallax pixel
Yield value to each pixel of right parallax pixel
In addition, carry out parallax modulate when, it is also possible to take by each parallax pixel in the pixel arrangement on imaging apparatus that
The geometric average that the density ratio of the RGB between this is taken into account.That is, it is R between left parallax pixel：G：B=1：2：1,
Also it is R between right parallax pixel：G：B=1：2：1, therefore give adding for 1/4 power to the parallax modulation based on R component
Power, to the parallax modulation based on G component the weighting of 1/2 power is given, and to the parallax modulation based on B component 1/4 power is given
Weighting, using the distribution paid attention to based on the parallax modulation of the high G component of density.Following a) illustrates the point on the basis of arithmetic average
Parallax modulation.
A) the parallax modulation put on the basis of arithmetic average
Left view difference modulation
Right parallax modulation
Following b) the parallax modulation put on the basis of geometric average is shown.
B) the parallax modulation put on the basis of geometric average
Left view difference modulation
Right parallax modulation
Above formula is rewritten, then
Left view difference modulation
Right parallax modulation
Additionally, in the 4th embodiment, the mode of the hardware of the last description for being suitable for the 3rd embodiment is illustrated, last
The step of 7 in the shading value amendment of anaglyph is carried out to the anaglyph of high exploring.But, in order to be obtained by software processing
The image of higher quality, it is also possible to the shading value amendment of anaglyph is inserted between step 3 and step 4, to interim parallax
Pixel carries out shading value amendment.Thus, complete white the regarding by step 6 of the GTG produced during the gain of shading value amendment is lifted is played
The effect that difference modulation makes up.
＜ supplements ＞
Situation to shading value amendment described above to be applied to dynamic image is illustrated.It is dynamic in input picture
In the case of image, computing as the 3rd embodiment can be carried out to a width width two field picture.In the above-described embodiment, take
Multiple stabilisation schemes of impact will not be brought on the picture structure of subject and noise etc..But, if desired dynamic figure
As the scheme of time shaft more stableization for shading value amendment, then it is also required to consider as follows.In dynamic image photography, camera
Side shifts near, is moved away from slow in one's movementsly, or the situation for making f-number lentamente change and being shot is universal.That is, as
Stereo-picture shading value distribution change will because pupil location and pupil diameter it is slowly varying.Exist mostly in dynamic image
The initial configuration I pictures (picture) that scene is checked, intermediate configurations P picture, B pictures.Therefore, it can from an I picture under
During till the frame nearby of one I pictures, by the step of obtaining in the 3rd embodiment 5 one-dimensional gain profiles data relative to
Time-axis direction lentamente carries out Function Fitting.
Also the equipment such as personal computer can be made as undertaking image data generating section 530, One Dimensional Projection data generating section
531st, one-dimensional smoothed data generating unit 532, one-dimensional reference data generating unit 533, the image of the function of shading value correction portion 534
Processing meanss and function.Image processing apparatus can be taken into parallax image data by other devices from camera etc..In this case,
Image data generating section 530 undertakes the effect as image data acquisition part.Additionally, image data generating section 530 oneself is produced
In the case of parallax image data, it is also possible to say it is to generate by self and obtain parallax image data.Image processing apparatus are not
It is limited to personal computer, various modes can be adopted.For example, TV, mobile phone, game machine etc. have display part or
The equipment being connected with display part can be image processing apparatus.Additionally, in the above description, there is image and refer to picture number
According to situation, there is also image refer in accordance with form launch and visual subject as the situation of itself.
In addition, in above-mentioned each embodiment, as the acquisition side of the parallax information of the pupil cutting based on simple eye formula
Method, it is also possible to consider following such variation.
1) incident light for making monocular lens is separated by way of prism is divided equally with luminous intensity to both direction, by respectively
The left view difference imaging apparatus and right parallax imaging apparatus of unilateral parallax pixel are only paved with carrying out double-plate shooting.Thereby, it is possible to
Make each pixel that there are right parallax pixel value and left parallax pixel value both information.
2) as the structure of monocular lens, the aperture suitable with the hypothetical pupil shown in Fig. 4 can be interleaved alternately with right side
And left side, for the black and white imaging apparatus of common no-parallax pixel, sequentially obtain left view difference image to double expose and the right side regards
Difference image.
More than, the present invention is illustrated using embodiment, but the technical scope of the present invention is not limited to above-mentioned embodiment
The scope of record.Clearly can apply numerous variations to above-mentioned embodiment to those skilled in the art or change
It is good.Knowable to claims, the mode after applying such change or improveing can also be contained in the technical scope of the present invention.
Description of reference numerals
10 digital cameras, 20 photographic lens, 21 optical axises, 50 digital cameras, 60 photographic lens, 61 optical axises, 100 shooting units
Part, 101 lenticules, 102 colored filters, 103 aperture masks, 104 opening portions, 105 wiring layers, 106 wirings, 107 openings,
108 photo-electric conversion elements, 109 substrates, 201 control units, 202A/D change-over circuits, 203 memories, 204 drive divisions, 205 images
Processing unit, 207 storage card IF, 208 operating portions, 209 display parts, 210LCD drive circuits, 220 storage cards, the life of 230 view data
Into portion, 231 smoothed data generating units, 232 destination image data generating units, 233 gain-boosted op amp portions, 322 center lines, 400 take the photograph
Element, 501 control units, 502A/D change-over circuits, 503 memories, 504 drive divisions, 505 image processing parts, 507 storage cards
IF, 508 operating portions, 509 display parts, 510LCD drive circuits, 520 storage cards, 530 image data generating sections, 531 One Dimensional Projections
Data generating section, 532 one-dimensional smoothed data generating units, 533 one-dimensional reference data generating units, 534 shading value correction portions, 1801
Distribution curve, 1802 distribution curves, 1803 distribution curves, 1804 distribution curves, 1805 distribution curves, 1806 synthesis distributions are bent
Line, 1807 distribution curves, 1808 distribution curves, 1809 synthesis distribution curves.
Claims (27)
1. a kind of image processing apparatus, it is characterised in that have：
Be input into by an optical system to whole light beams in the segment beam passed through from region different from each other object
Body image shot obtained from the 1st viewpoint direction image and the 2nd viewpoint direction image at least two anaglyphs
Mechanism；
The image of image and the 2nd viewpoint direction to the 1st viewpoint direction is respectively at least regarding the 1st viewpoint and the 2nd
Apply smoothing wave filter on the direction of point connection, the 1st machine for smoothing image and the 2nd smoothing image is thus generated respectively
Structure；
Generate by obtained from geometric average of the value based on the described 1st smoothing image with the value of the described 2nd smoothing image
The mechanism of the benchmark image that value is constituted；
By the way that the value of the benchmark image is generated into the machine of the 1st gain profiles data divided by the value of the described 1st smoothing image
Structure；
The mechanism of the gain-boosted op amp of each pixel is carried out to the image of the 1st viewpoint direction based on the 1st gain profiles data；
And
The image for carrying out the 1st viewpoint direction after the gain-boosted op amp is exported as the anaglyph for stereoscopic display
Mechanism,
The smoothing wave filter be have with the image of the image in the 1st viewpoint direction and the 2nd viewpoint direction it
Between the parallax amount that produces for the filtering diameter of same degree the wave filter that smooths of carrying out.
2. image processing apparatus as claimed in claim 1, it is characterised in that also have：
By the way that the value of the benchmark image is generated into the machine of the 2nd gain profiles data divided by the value of the described 2nd smoothing image
Structure；
The mechanism of the gain-boosted op amp of each pixel is carried out to the image of the 2nd viewpoint direction based on the 2nd gain profiles data；
The image of the 1st viewpoint direction after the gain-boosted op amp will be carried out and carry out the 2nd viewpoint direction after the gain-boosted op amp
Image merge and the mechanism that exports as the anaglyph for being used for stereoscopic display.
3. image processing apparatus as claimed in claim 1, it is characterised in that
With obtain the 1st viewpoint direction interim anaglyph and the 2nd viewpoint direction interim anaglyph and with than
The high high exploring of the interim anaglyph difference obtains the reference direction between the 1st viewpoint direction and the 2nd viewpoint direction
Benchmark image, and using the interim anaglyph of the 1st viewpoint direction, the interim disparity map of the 2nd viewpoint direction
Picture and the benchmark image generate the parallax of the high exploring of the anaglyph and the 2nd viewpoint direction of the high exploring of the 1st viewpoint direction
During the mechanism of image,
The interim anaglyph of the 1st viewpoint direction is input into as the image of the 1st viewpoint direction, as described
The image of 2 viewpoint directions and be input into the interim anaglyph of the 2nd viewpoint direction.
4. a kind of image processing apparatus, it is characterised in that have：
Be input into by an optical system to whole light beams in the segment beam passed through from region different from each other object
Body image shot obtained from the 1st viewpoint direction image and the 2nd viewpoint direction image at least two anaglyphs
Mechanism；
The image of image and the 2nd viewpoint direction to the 1st viewpoint direction is respectively at least regarding the 1st viewpoint and the 2nd
Apply smoothing wave filter on the direction of point connection, the 1st machine for smoothing image and the 2nd smoothing image is thus generated respectively
Structure；
It is worth obtained from the geometric average of the value for generating the value based on the described 1st smoothing image and the 2nd smoothing image
The mechanism of the benchmark image of composition；
By the way that the value of the benchmark image is generated into the machine of the 1st gain profiles data divided by the value of the described 1st smoothing image
Structure；
By the way that the value of the benchmark image is generated into the machine of the 2nd gain profiles data divided by the value of the described 2nd smoothing image
Structure；
The mechanism of the gain-boosted op amp of each pixel is carried out to the image of the 1st viewpoint direction based on the 1st gain profiles data；
The mechanism of the gain-boosted op amp of each pixel is carried out to the image of the 2nd viewpoint direction based on the 2nd gain profiles data；
The image of the 1st viewpoint direction after the gain-boosted op amp will be carried out and the 2nd viewpoint direction after the gain-boosted op amp is carried out
Image merge and the mechanism that exports as the anaglyph for being used for stereoscopic display；And
Adjusted between the anaglyph for stereoscopic display by making the filtering diameter of the smoothing wave filter variable
Parallax amount mechanism,
The filtering diameter is set to contract when parallax is eliminated to zero, when parallax is preserved and in two parallax pictures of the input picture
The maximum disparity amount that can be produced between element is same degree or is extended to more than it.
5. image processing apparatus as claimed in claim 4, it is characterised in that
With obtain the 1st viewpoint direction interim anaglyph and the 2nd viewpoint direction interim anaglyph and with than
The high high exploring of the interim anaglyph difference obtains the reference direction between the 1st viewpoint direction and the 2nd viewpoint direction
Benchmark image, and using the interim anaglyph of the 1st viewpoint direction, the interim disparity map of the 2nd viewpoint direction
Picture and the benchmark image generate the parallax of the high exploring of the anaglyph and the 2nd viewpoint direction of the high exploring of the 1st viewpoint direction
During the mechanism of image,
The interim anaglyph of the 1st viewpoint direction is input into as the image of the 1st viewpoint direction, as described
The image of 2 viewpoint directions and be input into the interim anaglyph of the 2nd viewpoint direction.
6. a kind of image processing apparatus, it is characterised in that have：
Image data acquisition part, it is obtained corresponding to the 1st viewpoint obtained by the common optical system of at least a portion
The 2nd view data corresponding to 1st view data and the 2nd viewpoint；
Smoothed data generating unit, it, will be by the 1st figure respectively to the 1st view data and the 2nd view data
As the mode that the picture of data and the 2nd pictorial data representation is respectively relative to parallax directions smoothing is filtered, thus give birth to
Into the 1st smoothed data and the 2nd smoothed data；
Destination image data generating unit, it uses the 1st smoothed data and each corresponding to the 2nd smoothed data
The geometric average of pixel value is generating destination image data；And
Gain-boosted op amp portion, it corrects the described 1st using the destination image data relative to the ratio of the 1st smoothed data
View data.
7. image processing apparatus as claimed in claim 6, it is characterised in that
Described is corrected using the destination image data relative to the ratio of the 2nd smoothed data in the gain-boosted op amp portion
2 view data.
8. image processing apparatus as claimed in claims 6 or 7, it is characterised in that
The parallax amount that the smoothed data generating unit will be produced between the 1st view data and the 2nd view data
Smoothed as amplitude of samples.
9. image processing apparatus as claimed in claims 6 or 7, it is characterised in that
Described image data acquiring section obtains the reference image data corresponding to the 3rd viewpoint as reference direction,
With high exploring image data generating section, it uses the 1st view data, the 2nd view data and the benchmark
View data generating high exploring reference image data, using the high exploring reference image data and the revised described 1st
View data generating the view data of high exploring the 1st, using the high exploring reference image data and revised 2nd figure
The view data of high exploring the 2nd is generated as data.
10. a kind of image processing apparatus, it is characterised in that have：
Image data acquisition part, it is obtained corresponding to the 1st viewpoint obtained by the common optical system of at least a portion
The 2nd view data corresponding to 1st view data and the 2nd viewpoint；
Smoothed data generating unit, it, will be by the 1st figure respectively to the 1st view data and the 2nd view data
As the mode that the picture of data and the 2nd pictorial data representation is respectively relative to parallax directions smoothing is filtered, thus give birth to
Into the 1st smoothed data and the 2nd smoothed data；
Destination image data generating unit, it uses the 1st smoothed data and each corresponding to the 2nd smoothed data
The geometric average of pixel value is generating destination image data；
Gain-boosted op amp portion, it corrects the described 1st using the destination image data relative to the ratio of the 1st smoothed data
View data, and using the destination image data relative to the ratio of the 2nd smoothed data correcting the 2nd image
Data；And
Parallax amount adjustment portion, its amplitude of samples for passing through to adjust the smoothing is corrected to adjust based on the gain-boosted op amp portion
Parallax amount between rear the 1st view data and the 2nd view data.
11. image processing apparatus as claimed in claim 10, it is characterised in that
In the case of the parallax amount inscribed when maintenance is obtained by described image data acquiring section, the parallax amount adjustment
Portion is adjusted to the amplitude of samples more than the maximum disparity amount of the 1st view data and the 2nd view data, is reducing
By described image data acquiring section obtain when the parallax amount inscribed in the case of, the amplitude of samples is adjusted to into not enough
The maximum disparity amount of the 1st view data and the 2nd view data.
12. image processing apparatus as described in claim 10 or 11, it is characterised in that
Described image data acquiring section obtains the reference image data corresponding to the 3rd viewpoint as reference direction,
With high exploring image data generating section, it uses the 1st view data, the 2nd view data and the benchmark
View data generating high exploring reference image data, using the high exploring reference image data and the revised described 1st
View data generating the view data of high exploring the 1st, using the high exploring reference image data and revised 2nd figure
The view data of high exploring the 2nd is generated as data.
A kind of 13. camera heads, have：
Imaging apparatus；With
Image processing apparatus any one of claim 6 to 12,
The camera head is characterised by,
1st view data and the 2nd view data are based on the output of the imaging apparatus and generate.
14. a kind of image processing methods, it is characterised in that perform following steps：
Image data acquisition step, is obtained corresponding to the 1st viewpoint obtained by the common optical system of at least a portion
The 2nd view data corresponding to 1st view data and the 2nd viewpoint；
Smoothed data generation step, respectively to the 1st view data and the 2nd view data, will be by the 1st figure
As the mode that the picture of data and the 2nd pictorial data representation is respectively relative to parallax directions smoothing is filtered, thus give birth to
Into the 1st smoothed data and the 2nd smoothed data；
Destination image data generation step, using each corresponding to the 1st smoothed data and the 2nd smoothed data
The geometric average of pixel value is generating destination image data；And
Gain-boosted op amp step, the described 1st is corrected using the destination image data relative to the ratio of the 1st smoothed data
View data.
15. a kind of image processing apparatus, it is characterised in that have：
Be input into by an optical system to whole light beams in the segment beam passed through from region different from each other object
Body image shot obtained from the 1st viewpoint direction image and the 2nd viewpoint direction image at least two anaglyphs
Mechanism；
The image of image and the 2nd viewpoint direction to the 1st viewpoint direction, will regard respectively in connection the 1st viewpoint and the 2nd
The pixel value of the multiple pixels arranged on the orthogonal y directions in x directions of point is added, and generates the 1st after One Dimensional Projection respectively
The mechanism of the one-dimensional distributed data of viewpoint direction and the one-dimensional distributed data of the 2nd viewpoint direction；
The one-dimensional distributed data of one-dimensional distributed data and the 2nd viewpoint direction to the 1st viewpoint direction after the One Dimensional Projection point
Do not apply one-dimensional smoothing wave filter, the 1st one-dimensional smoothing distributed data and the 2nd one-dimensional smoothing distribution are thus generated respectively
The mechanism of data；
Obtain the described 1st one-dimensional arithmetic average or geometry for smoothing distributed data smoothing distributed data one-dimensional with the described 2nd
Mechanism that is average and generating one-dimensional benchmark distributed data；
By the way that the value of the one-dimensional benchmark distributed data is generated into the 1st divided by the value of the described 1st one-dimensional smoothing distributed data
The mechanism of one-dimensional gain profiles data；
Image to the 1st viewpoint direction, the described 1st one-dimensional gain profiles data are commonly used with each y-coordinate, and are divided
The mechanism of the gain-boosted op amp of the one-dimensional square is not carried out；And
The image for carrying out the 1st viewpoint direction after the gain-boosted op amp is exported as the anaglyph for stereoscopic display
Mechanism,
The one-dimensional smoothing wave filter is the image in the image than the 1st viewpoint direction with the 2nd viewpoint direction
Between the wave filter that smoothed in the big scope of the maximum disparity amount that can produce.
16. image processing apparatus as claimed in claim 15, it is characterised in that also have：
By the way that the value of the one-dimensional benchmark distributed data is generated into the 2nd divided by the value of the described 2nd one-dimensional smoothing distributed data
The mechanism of one-dimensional gain profiles data；
Image to the 2nd viewpoint direction, the described 2nd one-dimensional gain profiles data are commonly used in each row, and are divided
The mechanism of the gain-boosted op amp of the one-dimensional square is not carried out；And
The image of the 1st viewpoint direction after the gain-boosted op amp will be carried out and the 2nd viewpoint direction after the gain-boosted op amp is carried out
Image merge and the mechanism that exports as the anaglyph for being used for stereoscopic display.
17. image processing apparatus as claimed in claim 16, it is characterised in that
The 1st one-dimensional gain profiles data for obtaining further are entered by the mechanism for generating the described 1st one-dimensional gain profiles data
Line function is fitted and approximate with gentle change curve, and using the matched curve as carrying out the image of the 1st viewpoint direction
Gain-boosted op amp one-dimensional gain profiles data and use,
The mechanism for generating the described 2nd one-dimensional gain profiles data further enters to the 2nd one-dimensional gain profiles data for obtaining
Line function is fitted and approximate with gentle change curve, and using the matched curve as carrying out the image of the 2nd viewpoint direction
Gain-boosted op amp one-dimensional gain profiles data and use.
18. image processing apparatus as claimed in claim 17, it is characterised in that
The Function Fitting is approximate by 4 order polynomials.
19. image processing apparatus as claimed in claim 15, it is characterised in that
The image of the 1st viewpoint direction and the image of the 2nd viewpoint direction are the arbitrary optics in the optical system
Under the conditions of shoot the arbitrary common subject image for obtaining.
20. image processing apparatus as any one of claim 15 to 18, it is characterised in that
One-dimensional data and the one-dimensional distribution of the 2nd viewpoint direction for carrying out the 1st viewpoint direction after the One Dimensional Projection is generated respectively
The mechanism of data, the thumbnail image of the image of the 1st viewpoint direction and the image of the 2nd viewpoint direction are used as input picture
Thumbnail image.
21. a kind of image processing apparatus, it is characterised in that have：
Image data acquisition part, it is obtained corresponding to the 1st viewpoint obtained by the common optical system of at least a portion
The 2nd view data corresponding to 1st view data and the 2nd viewpoint；
One Dimensional Projection data generating section, its respectively to the 1st view data and the 2nd view data, will with parallax side
The pixel value of the multiple pixels arranged on orthogonal direction is added, and thus generates the 1st One Dimensional Projection data and the 2nd One Dimensional Projection
Data；
One-dimensional smoothed data generating unit, its respectively to the 1st One Dimensional Projection data and the 2nd One Dimensional Projection data, with
Amplitude of samples more than maximum disparity amount calculates rolling average, thus generates the 1st one-dimensional smoothed data and the 2nd one-dimensional smooth
Change data；
One-dimensional reference data generating unit, it uses the 1st one-dimensional smoothed data and the 2nd one-dimensional smoothed data institute right
The arithmetic average or geometric average of each pixel value answered is generating one-dimensional reference data；And
Shading value correction portion, it uses the one-dimensional reference data described relative to the comparison of the 1st one-dimensional smoothed data
1st view data carries out shading value amendment.
22. image processing apparatus as claimed in claim 21, it is characterised in that
The shading value correction portion is described relative to the comparison of the 2nd one-dimensional smoothed data using the one-dimensional reference data
2nd view data carries out shading value amendment.
23. image processing apparatus as claimed in claim 22, it is characterised in that
The shading value correction portion generates the one-dimensional reference data using Function Fitting relative to the 1st one-dimensional smoothing
The ratio of the ratio of data and the one-dimensional reference data relative to the 2nd one-dimensional smoothed data.
24. image processing apparatus as claimed in claim 23, it is characterised in that
Function is 4 order polynomials.
25. image processing apparatus as any one of claim 21 to 24, it is characterised in that
Described image data acquiring section makes the exploring of each picture by the 1st view data and the 2nd pictorial data representation
Degree is reduced,
The One Dimensional Projection data generating section respectively to represent resolution reduce after picture the 1st view data and described the
2 view data, generate the 1st One Dimensional Projection data and the 2nd One Dimensional Projection data.
A kind of 26. camera heads, have：
Imaging apparatus；With
Image processing apparatus any one of claim 21 to 25,
The camera head is characterised by,
1st view data and the 2nd view data are based on the output of the imaging apparatus and generate.
27. a kind of image processing methods, it is characterised in that perform following steps：
Image data acquisition step, is obtained corresponding to the 1st viewpoint obtained by the common optical system of at least a portion
The 2nd view data corresponding to 1st view data and the 2nd viewpoint；
One Dimensional Projection data genaration step, respectively to the 1st view data and the 2nd view data, will with parallax side
The pixel value of the multiple pixels arranged on orthogonal direction is added, and thus generates the 1st One Dimensional Projection data and the 2nd One Dimensional Projection
Data；
One-dimensional smoothed data generation step, respectively to the 1st One Dimensional Projection data and the 2nd One Dimensional Projection data, with
Amplitude of samples more than maximum disparity amount calculates rolling average, thus generates the 1st one-dimensional smoothed data and the 2nd one-dimensional smooth
Change data；
One-dimensional reference data generation step is right using the 1st one-dimensional smoothed data and the 2nd one-dimensional smoothed data institute
The arithmetic average or geometric average of each pixel value answered is generating one-dimensional reference data；And
Shading value amendment step, it is described relative to the comparison of the 1st one-dimensional smoothed data using the one-dimensional reference data
1st view data carries out shading value amendment.
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2013075018 | 2013-03-29 | ||
| JP2013075092 | 2013-03-29 | ||
| JP2013-075018 | 2013-03-29 | ||
| JP2013-075092 | 2013-03-29 | ||
| PCT/JP2014/001884 WO2014156202A1 (en) | 2013-03-29 | 2014-03-31 | Image processing device, image capture device, and image processing program |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105122793A CN105122793A (en) | 2015-12-02 |
| CN105122793B true CN105122793B (en) | 2017-05-10 |
Family
ID=51623212
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201480018899.8A Active CN105122793B (en) | 2013-03-29 | 2014-03-31 | Image processing device, image capture device, and image processing program |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US9942537B2 (en) |
| JP (1) | JP6048574B2 (en) |
| CN (1) | CN105122793B (en) |
| WO (1) | WO2014156202A1 (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013190814A1 (en) * | 2012-06-22 | 2013-12-27 | 株式会社ニコン | Image processing device, imaging device, and image processing program |
| WO2014192300A1 (en) | 2013-05-31 | 2014-12-04 | 株式会社ニコン | Imaging element, imaging device, and image processing device |
| WO2016002578A1 (en) * | 2014-07-04 | 2016-01-07 | ソニー株式会社 | Image processing device and method |
| JP6459482B2 (en) * | 2014-12-18 | 2019-01-30 | 株式会社リコー | Parallax value deriving apparatus, device control system, moving body, robot, parallax value deriving method, and program |
| JP6735621B2 (en) * | 2016-07-21 | 2020-08-05 | キヤノン株式会社 | Image processing apparatus, control method thereof, program, and imaging apparatus |
| EP3261328B1 (en) | 2016-06-03 | 2021-10-13 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and computer-readable storage medium |
| CN107734214A (en) * | 2016-08-10 | 2018-02-23 | 宁波舜宇光电信息有限公司 | With the multi-cam module of different size aperture and its application |
| US11012605B2 (en) * | 2016-11-08 | 2021-05-18 | Sony Corporation | Image processing device, image processing method, and program for generating a focused image |
| JP6862210B2 (en) * | 2017-02-16 | 2021-04-21 | キヤノン株式会社 | Image processing equipment, image processing system, image processing method and program |
| WO2019202984A1 (en) * | 2018-04-17 | 2019-10-24 | 富士フイルム株式会社 | Image capturing device, distance measuring method, distance measuring program, and recording medium |
| US11323682B2 (en) * | 2018-07-02 | 2022-05-03 | Sony Interactive Entertainment Inc. | Electronic device, content processing device, content processing system, image data output method, and image processing method |
| US10937244B2 (en) * | 2018-10-23 | 2021-03-02 | Microsoft Technology Licensing, Llc | Efficiency enhancements to construction of virtual reality environments |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012036019A1 (en) * | 2010-09-13 | 2012-03-22 | 富士フイルム株式会社 | Monocular 3d-imaging device, shading correction method for monocular 3d-imaging device, and program for monocular 3d-imaging device |
| WO2012039346A1 (en) * | 2010-09-22 | 2012-03-29 | 富士フイルム株式会社 | Stereoscopic image capture device and shading correction method |
| CN102870422A (en) * | 2011-02-15 | 2013-01-09 | 富士胶片株式会社 | Image capturing device, image capturing device main body, and shading correction method |
| WO2013042440A1 (en) * | 2011-09-21 | 2013-03-28 | 富士フイルム株式会社 | Image processing device, method, program and recording medium, stereoscopic image capture device, portable electronic apparatus, printer, and stereoscopic image player device |
Family Cites Families (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10232626A (en) * | 1997-02-20 | 1998-09-02 | Canon Inc | Stereoscopic image display device |
| JP2003007994A (en) | 2001-06-27 | 2003-01-10 | Konica Corp | Solid-state image pickup element, stereoscopic camera apparatus, and range finder |
| US8131098B2 (en) * | 2007-07-06 | 2012-03-06 | Panasonic Corporation | Image processing device, image processing method, image processing system, program, storage medium, and integrated circuit |
| WO2009066770A1 (en) | 2007-11-22 | 2009-05-28 | Nikon Corporation | Digital camera and digital camera system |
| US8866920B2 (en) * | 2008-05-20 | 2014-10-21 | Pelican Imaging Corporation | Capturing and processing of images using monolithic camera array with heterogeneous imagers |
| US7948514B2 (en) * | 2008-06-02 | 2011-05-24 | Panasonic Corporation | Image processing apparatus, method and computer program for generating normal information, and viewpoint-converted image generating apparatus |
| JP5237077B2 (en) | 2008-12-16 | 2013-07-17 | キヤノン株式会社 | FOCUS DETECTION DEVICE, ITS CONTROL METHOD, AND PROGRAM |
| US8514491B2 (en) * | 2009-11-20 | 2013-08-20 | Pelican Imaging Corporation | Capturing and processing of images using monolithic camera array with heterogeneous imagers |
| WO2011114572A1 (en) * | 2010-03-19 | 2011-09-22 | 富士フイルム株式会社 | Imaging device, method and program, and recording medium using same |
| JP2011223562A (en) * | 2010-03-23 | 2011-11-04 | Fujifilm Corp | Imaging apparatus |
| JP5595505B2 (en) * | 2010-09-10 | 2014-09-24 | 富士フイルム株式会社 | Stereo imaging device and stereo imaging method |
| WO2012132797A1 (en) * | 2011-03-31 | 2012-10-04 | 富士フイルム株式会社 | Image capturing device and image capturing method |
| WO2013027513A1 (en) * | 2011-08-19 | 2013-02-28 | 富士フイルム株式会社 | Image pickup apparatus and shading correction method |
| JP6019568B2 (en) * | 2011-11-28 | 2016-11-02 | ソニー株式会社 | Image processing apparatus and method, recording medium, and program |
| WO2013099169A1 (en) * | 2011-12-27 | 2013-07-04 | パナソニック株式会社 | Stereo photography device |
| JP6131546B2 (en) | 2012-03-16 | 2017-05-24 | 株式会社ニコン | Image processing apparatus, imaging apparatus, and image processing program |
| CN104185983B (en) | 2012-03-16 | 2016-10-26 | 株式会社尼康 | Imaging apparatus, camera head and camera system |
| JP6131545B2 (en) | 2012-03-16 | 2017-05-24 | 株式会社ニコン | Image processing apparatus, imaging apparatus, and image processing program |
| EP2833638B1 (en) * | 2012-03-29 | 2017-09-27 | Fujifilm Corporation | Image processing device, imaging device, and image processing method |
| WO2013161313A1 (en) | 2012-04-25 | 2013-10-31 | 株式会社ニコン | Image processing device, imaging device, and image processing program |
| WO2013190814A1 (en) | 2012-06-22 | 2013-12-27 | 株式会社ニコン | Image processing device, imaging device, and image processing program |
| CN104521231B (en) | 2012-08-10 | 2016-12-21 | 株式会社尼康 | Image processing apparatus and camera head |
| EP2898689B1 (en) * | 2012-09-21 | 2020-05-06 | Nokia Technologies Oy | Method and apparatus for video coding |
-
2014
- 2014-03-31 CN CN201480018899.8A patent/CN105122793B/en active Active
- 2014-03-31 WO PCT/JP2014/001884 patent/WO2014156202A1/en active Application Filing
- 2014-03-31 JP JP2015508097A patent/JP6048574B2/en active Active
-
2015
- 2015-09-29 US US14/869,527 patent/US9942537B2/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012036019A1 (en) * | 2010-09-13 | 2012-03-22 | 富士フイルム株式会社 | Monocular 3d-imaging device, shading correction method for monocular 3d-imaging device, and program for monocular 3d-imaging device |
| WO2012039346A1 (en) * | 2010-09-22 | 2012-03-29 | 富士フイルム株式会社 | Stereoscopic image capture device and shading correction method |
| CN102870422A (en) * | 2011-02-15 | 2013-01-09 | 富士胶片株式会社 | Image capturing device, image capturing device main body, and shading correction method |
| WO2013042440A1 (en) * | 2011-09-21 | 2013-03-28 | 富士フイルム株式会社 | Image processing device, method, program and recording medium, stereoscopic image capture device, portable electronic apparatus, printer, and stereoscopic image player device |
Also Published As
| Publication number | Publication date |
|---|---|
| US20160094835A1 (en) | 2016-03-31 |
| JP6048574B2 (en) | 2016-12-21 |
| CN105122793A (en) | 2015-12-02 |
| JPWO2014156202A1 (en) | 2017-02-16 |
| WO2014156202A1 (en) | 2014-10-02 |
| US9942537B2 (en) | 2018-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105122793B (en) | Image processing device, image capture device, and image processing program | |
| CN107925751B (en) | System and method for multiple views noise reduction and high dynamic range | |
| CN103826033B (en) | Image processing method, image processing equipment, image pick up equipment and storage medium | |
| US9432656B2 (en) | Image capturing device including lens array and processing | |
| CN102917235B (en) | Image processing apparatus and image processing method | |
| CN103688536B (en) | Image processing apparatus, image processing method | |
| US9749601B2 (en) | Imaging device, image display method, and storage medium for displaying reconstruction image | |
| CN104429056B (en) | Image processing method, image processing apparatus, camera device and image processing program | |
| CN106165398B (en) | Photographing element, photographic device and image processing apparatus | |
| CN104521231B (en) | Image processing apparatus and camera head | |
| CN104662887B (en) | Image processing equipment, image processing method and the picture pick-up device with the image processing equipment | |
| CN101588439B (en) | Method for correcting multi-anaglyph color cast | |
| CN103718547B (en) | Image processing equipment, image processing method and program | |
| CN104144284B (en) | Imaging device and imaging method | |
| CN107105216B (en) | A kind of 3 d light fields display device of continuous parallax based on pinhole array, wide viewing angle | |
| CN104170377B (en) | Image processing apparatus, camera head | |
| US9832404B2 (en) | Image sensor, imaging apparatus, and image processing device | |
| CN104322061B (en) | Image processing device, imaging device, and image processing method | |
| WO2011099896A1 (en) | Method for representing an initial three-dimensional scene on the basis of results of an image recording in a two-dimensional projection (variants) | |
| CN104168403A (en) | High dynamic range video recording method and device based on Bayer color filtering array | |
| CN105939439B (en) | Image processing apparatus, photographic device and image processing method | |
| CN105282534B (en) | For being embedded in the system and method for stereo-picture | |
| CN103503447B (en) | The control method of filming apparatus and filming apparatus | |
| TWI504936B (en) | Image processing device | |
| JP6439285B2 (en) | Image processing apparatus, imaging apparatus, and image processing program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |