JP2009181367A - Imaging apparatus and electronic apparatus - Google Patents

Abstract

An image pickup apparatus and an electronic apparatus capable of simplifying an optical system and reducing costs, and capable of obtaining a good restored image with appropriate image quality corresponding to various uses and shooting conditions. provide.
The shape of the modulation surface 213a of the light wavefront modulation element 213A and the shape of the diaphragm 214 are rotationally symmetrical and continuous with respect to the optical axis, and the light wavefront modulation function is manifested or not manifested, or the light wavefront modulation characteristics. The diaphragm 214 is provided with an adjustment function so that switching can be performed.
[Selection] Figure 5

Description

  The present invention relates to an imaging apparatus and an electronic apparatus that include an imaging device and includes an optical system.

In response to the digitization of information, which has been rapidly developing in recent years, the response in the video field is also remarkable.
In particular, as symbolized by a digital camera, a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensor, which is a solid-state image sensor, is used in most cases instead of a conventional film.

  As described above, an imaging lens device using a CCD or CMOS sensor as an imaging element is for taking an image of a subject optically by an optical system and extracting it as an electrical signal by the imaging element. In addition to a digital still camera, It is used in video cameras, digital video units, personal computers, mobile phones, personal digital assistants (PDAs), image inspection devices, industrial cameras for automatic control, and the like.

FIG. 25 is a diagram schematically illustrating a configuration and a light flux state of a general imaging lens device.
The imaging lens device 1 includes an optical system 2 and an imaging element 3 such as a CCD or CMOS sensor.
In the optical system, the object side lenses 21 and 22, the diaphragm 23, and the imaging lens 24 are sequentially arranged from the object side (OBJS) toward the image sensor 3 side.

In the imaging lens device 1, as shown in FIG. 25, the best focus surface is matched with the imaging element surface.
FIGS. 26A to 26C show spot images on the light receiving surface of the imaging element 3 of the imaging lens device 1.

In addition, imaging devices have been proposed in which light beams are regularly dispersed by a phase plate and restored by digital processing to enable imaging with a deep depth of field (for example, Non-Patent Documents 1 and 2 and Patent Documents). 1-5).
In addition, an automatic exposure control system for a digital camera that performs filter processing using a transfer function has been proposed (see, for example, Patent Document 6).

In addition, in an apparatus having an image input function such as a CCD or CMOS, it is often very useful to read a close still image such as a barcode together with a desired image such as a landscape.
For barcode reading, for example, as a first example, a technique for focusing by auto-focusing that extends a lens, and as a second example as a depth expansion technique, the depth of field is expanded by, for example, reducing the F value in a camera. Some have fixed focus.
Furthermore, a method for increasing the in-focus field is disclosed in Patent Document 8, for example.

“Wavefront Coding; jointly optimized optical and digital imaging systems”, Edward R. Dowski, Jr., Robert H. Cormack, Scott D. Sarama. “Wavefront Coding; A modern method of achieving high performance and / or low cost imaging systems”, Edward R. Dowski, Jr., Gregory E. Johnson. USP 6,021,005 USP 6,642,504 USP 6,525,302 USP 6,069,738 JP 2003-235794 A JP 2004-153497 A JP 2004-37733 A JP 2002-27047 A

In the imaging devices proposed in the above-mentioned documents, all of them are based on the assumption that the PSF (Point-Spread-Function) when the above-described phase plate is inserted into a normal optical system is constant, When the PSF changes, it is extremely difficult to realize an image with a deep depth of field by convolution using a subsequent kernel.
Therefore, apart from a single-focus lens, a zoom system, an AF system, or the like has a great problem in adopting due to the high accuracy of the optical design and the associated cost increase.
In other words, in the conventional imaging apparatus, proper convolution calculation cannot be performed, and astigmatism and coma that cause a shift of a spot (SPOT) image at the time of wide or tele (Tele). Therefore, an optical design that eliminates various aberrations such as zoom chromatic aberration is required.
However, the optical design that eliminates these aberrations increases the difficulty of optical design, causing problems such as an increase in design man-hours, an increase in cost, and an increase in size of the lens.

  Further, the character required for an image is different between a code reader such as a barcode reader and general photography (digital camera). In particular, the difference becomes large when a depth extension optical system is used. For example, a code reader or the like needs a high contrast so that a regular arrangement can be resolved. If a certain level of contrast is obtained, the influence of image blur is small. On the other hand, in general photography, the effect of visual blur increases.

  The present invention is capable of simplifying an optical system and reducing costs, as well as an imaging apparatus and an electronic apparatus capable of obtaining a good restored image having an appropriate image quality corresponding to various applications and photographing conditions. Is to provide.

  An image pickup apparatus according to a first aspect of the present invention includes an optical system including a lens and a light wavefront modulation element, an image pickup element that picks up a subject image that has passed through the optical system, and a light flux that passes through the light wavefront modulation element. The restricting portion can be changed, and by changing the stop, the light wavefront modulation function is turned on and off, or the light wavefront modulation characteristic is switched.

  Preferably, the diaphragm includes a plurality of kinds of diaphragms having different limiting portions.

  Preferably, the aperture includes a variable aperture that makes the limiting portion variable.

  Preferably, an image processing apparatus that performs predetermined image processing on an image signal obtained from the image sensor, the image processing apparatus determines whether or not image processing image processing is performed or an image depending on a state of the aperture. Switch the processing contents.

  Preferably, in the stop, the light wavefront modulation element is disposed in the vicinity of the stop.

  Preferably, the light wavefront modulation function of the light wavefront modulation element and the shape of the diaphragm are substantially rotationally symmetric with respect to the optical axis.

  According to a second aspect of the present invention, there is provided an electronic apparatus having an imaging device, wherein the imaging device includes an optical system including a lens and a light wavefront modulation element, and an imaging element that captures a subject image that has passed through the optical system. A diaphragm that restricts a light beam passing through the light wavefront modulation element and can change the restricted portion, and by changing the diaphragm, the light wavefront modulation function is manifested or not manifested, or the light wavefront Switches the modulation characteristics.

  According to the present invention, not only can the optical system be simplified and the cost can be reduced, but also there is an advantage that a good restored image having an appropriate image quality can be obtained corresponding to various uses and photographing conditions. .

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is an external view showing an example of an information code reader as an electronic apparatus according to an embodiment of the present invention.
2A to 2C are diagrams illustrating examples of information codes.
FIG. 3 is a block diagram illustrating a configuration example of an imaging apparatus applicable to the information code reading apparatus of FIG.
Here, an information code reader is illustrated as an apparatus to which the imaging apparatus of the present embodiment is applicable.

As shown in FIG. 1, the information code reader 100 according to the present embodiment has a main body 110 connected to a processing device such as an electronic register (not shown) via a cable 111, for example, a reflectance printed on a reading object 120. It is a device that can read the information code 121 such as a different symbol or code.
As an information code to be read, for example, a one-dimensional bar code 122 such as a JAN code as shown in FIG. 2A, a stack-type CODE 49 as shown in FIGS. 2B and 2C, Alternatively, a two-dimensional barcode 123 such as a matrix type QR code can be used.

In the information code reading apparatus 100 according to the present embodiment, an illumination light source (not shown) and an imaging apparatus 200 as shown in FIG.
As will be described in detail later, the imaging apparatus 200 applies a light wavefront modulation element to the optical system, regularly disperses the light beam by the light wavefront modulation element, and restores the image by digital processing to capture an image with a deep depth of field. The system adopts a wavefront aberration control optical system or a depth expansion optical system (DEOS: Optical System) that makes possible a one-dimensional barcode such as a JAN code and a two-dimensional barcode such as a QR code. An information code such as a bar code can be accurately read with high accuracy.

  As shown in FIG. 3, the imaging device 200 of the information code reader 100 includes an optical system 210, an imaging device 220, an analog front end unit (AFE) 230, an image processing device 240, a camera signal processing unit 250, and an image display memory 260. , An image monitoring device 270, an operation unit 280, and a control device 290.

FIG. 4 is a diagram showing a basic configuration of the imaging lens unit forming the optical system according to the present embodiment.
The optical system 210 </ b> A supplies an image obtained by capturing the subject object OBJ to the image sensor 220. In the optical system 210A, a first lens 211, a second lens 212, a third lens 213, a diaphragm 214, a fourth lens 215, and a fifth lens 216 are arranged in this order from the object side.
In the optical system 210A of the present embodiment, a fourth lens 215 and a fifth lens 216 are connected. That is, the lens unit of the optical system 210A of the present embodiment is configured to include a cemented lens.

The optical system 210A of the present embodiment is configured as an optical system to which an optical wavefront modulation element is applied.
In the present embodiment, a light wave is applied to a depth extension optical system that uses an optical wavefront modulation element (phase modulation element) to make the change in OTF according to the object distance smaller than that of an optical system that does not have an optical wavefront modulation element. The surface modulation element is optimized.

In this embodiment, the light wavefront modulation element is formed so as to express a light wavefront modulation function alone.
The surface on the image sensor 220 side of the third lens 213 is configured to have a shape that exhibits a predetermined light wavefront modulation function.
A diaphragm 214 is disposed in the vicinity of the light wavefront modulation surface (for example, phase modulation surface) 213a.
The diaphragm 214 in the present embodiment includes a plurality of kinds of diaphragms or a variable diaphragm so that the light wavefront modulation function can be expressed and not expressed, or the light wavefront modulation characteristics can be switched.

In the example of FIG. 4, the central portion around the optical axis of the surface on the imaging surface side of the third lens 213 is formed in a concave shape with a predetermined curvature, and the third lens 213 is formed by this shape. It has the function of a light wavefront modulation element.
The light wavefront modulation surface 213a may have a shape capable of exhibiting the light wavefront modulation function even if it is formed in a convex shape instead of a concave shape.
In addition, an optical wavefront modulation surface is formed on the center side using an externally dependent optical wavefront modulation element such as a liquid crystal lens that depends on the outside whether the optical wavefront modulation function is manifested or not (for example, voltage application from the outside). However, it is also possible to provide an end side diaphragm function by forming a surface that restricts the passage of light on the end side.

  In this embodiment, the shape of the modulation surface 213a of the light wavefront modulation element 213 and the stop 214 are rotationally symmetrical with respect to the optical axis, and the light wavefront modulation function is manifested or not manifested, or the lightwave The diaphragm 214 is provided with an adjusting function so that the surface modulation characteristics can be switched.

Code readers such as barcode readers and general photography (digital cameras) have different properties required for images. In particular, the difference becomes large when a depth extension optical system is used.
For example, a code reader or the like needs a high contrast so that a regular arrangement can be resolved. If a certain level of contrast is obtained, the influence of image blur is small. On the other hand, in general photography, the effect of visual blur increases.
Therefore, in the present embodiment, the optical system taking into account the characteristics of a code reader or the like is set to a high contrast, and when the general image is obtained, the optical system suppresses the contrast and changes as a whole suitable for image restoration processing. The diaphragm 214 is configured to enable or disable the light wavefront modulation function or to switch the light wavefront modulation characteristics so that an MTF with less MTF can be obtained.

FIG. 5 is a diagram schematically showing the relationship between the light wavefront modulation element and the diaphragm according to the present embodiment.
In FIG. 5, the left side shows a case where the light beam passes only through the first phase plane PS1, and the right side shows a case where only the second phase plane PS2 passes.
6A shows the MTF when the light beam passes only through the first phase plane PS1 of FIG. 5, and FIG. 6B shows the MTF when it passes through the second phase plane PS2 of FIG. Showing. 7A shows the light intensity distribution when the light beam passes only through the first phase plane PS1 of FIG. 5, and FIG. 7B shows the light intensity distribution when the light beam passes through the second phase surface PS2 of FIG. The light intensity distribution is shown.
Here, the second phase plane PS means a second phase plane SP2 which is a wider plane including the first phase plane SP1.

In the present embodiment, the configuration of the modulation (phase) surface 213a of the light wavefront modulation element arranged in the optical system is a shape that emphasizes contrast near the center, and the MTF against the variation in distance in the peripheral portion. This constitutes a modulation surface in which fluctuations in the frequency are reduced.
By doing so, when the aperture diameter (aperture diameter) is reduced so that the light beam passes near the center of the lens, an image with high contrast is obtained by allowing the light to pass through a portion with little phase variation (change).
Further, if the aperture diameter (aperture diameter) is increased so that the light beam passes to the periphery of the lens, an image with less MTF fluctuation due to distance can be obtained by passing the light through a portion having a large phase fluctuation (change). As described above, the diaphragm at that time may be replaced with a diaphragm or the aperture diameter of the variable diaphragm may be increased.

In association with FIG. 5, there are a first phase plane PS1 that emphasizes contrast, and a second phase plane PS2 in which fluctuations in MTF with respect to distance are reduced.
As shown in FIGS. 6A and 6B, when the light beam passes only through the first phase plane PS1, the fluctuation of the MTF is gentle, and when the light beam passes through the second phase surface PS2, the MTF changes sharply. After that, the amount of change decreases.
Further, when the light beam passes only through the first phase plane PS1 even if the intensity distribution of the light beam is seen, as shown in FIG. 7A, it is higher (stronger) than the peak width, and conversely the second phase surface PS2. Is low (weak) with respect to the peak width, as shown in FIG. 7B.
By changing the image processing method based on this difference in tendency, it is possible to perform optimization in a situation requiring contrast and a situation requiring a general image.
Here, the former is considered to be effective for applications such as a code reader, and the latter is considered to be effective for photographing in a situation where a general subject is not selected.

In practical use, it can be applied to a camera built into a mobile phone, for example, and the aperture is controlled to the latter state in the normal shooting mode and to the latter state when the information code reading mode is set. It is possible to switch only by.
It is also possible to use a method of automatically switching the aperture control when the camera recognizes that the subject is an information code instead of switching the shooting mode.
When the imaging apparatus of this embodiment is used, the accuracy required for driving is not high compared to the normal method of switching by moving the optical system in the optical axis direction, and it is effective for suppressing the thickness of the optical system in the optical axis direction. It is.

In the above-described example, the method of changing the aperture diameter and switching the region through which the light beam passes is referred to as the first phase surface PS1 and the second phase surface PS2, but the surface having normal optical characteristics and the modulation (phase) ) A method of switching to a surface is also possible.
For example, the portion close to the optical axis center of the optical wavefront modulation element 213A through which the light beam passes when the aperture diameter is reduced (the region indicated as the first phase plane PS1 in FIG. 5) is a surface having no optical wavefront modulation function. When the aperture diameter is increased, the light wavefront modulation function can be realized by allowing the light beam to pass through a portion including the periphery thereof (a region indicated as the second phase plane PS2 in FIG. 5).
At this time, the content of the subsequent image processing also does not include the focus restoration processing (the dispersion image is made an image having no dispersion) when the aperture diameter is reduced and the light beam passes through the surface without the light wavefront modulation function. When the aperture diameter is increased and the light beam passes through a surface having a light wavefront modulation function, it is possible to perform switching so as to include a focus restoration process.

  Below, the structural example of a diaphragm is demonstrated.

  FIGS. 8A and 8B are diagrams showing examples of switching diaphragms. FIG. 8A shows an example of a disk rotation type switching diaphragm, and FIG. 8B shows a straight traveling type switching diaphragm. An example is shown.

A plurality of holes 2133 and 2134 are formed in the disc-shaped switching diaphragm plate 2131 or the plate-shaped switching diaphragm plate 2132, and the switching diaphragm plate 2131 or the plate-shaped switching diaphragm plate 2132 is moved in a direction perpendicular to the optical axis. Thus, the light flux passing through the hole for the light wavefront modulation element 213A is controlled.
In the case of a disk-shaped plate, the plurality of holes are switched by rotating, and in the case of a rectangular plate, it is slid in the longitudinal direction.
Although control of rotation and slide is not shown in the figure, for example, using a stepping motor as a power source, it is possible to drive the plate-like member by decelerating through a gear or the like.
The hole is formed so that the center (center of gravity) exists on the trajectory (on the one-dot chain line in the figure) through which the optical axis passes when rotating around the center of rotation in the case of a disk and when sliding on a rectangular plate. To do.

  FIG. 9 is a diagram illustrating an example of a variable aperture.

As shown in FIG. 9A, a plurality of (eight in the example of FIG. 9) diaphragm blades 2135 are arranged around the optical axis, and the rotation angle of each diaphragm blade is centered on the rotation center of the diaphragm blade 2135. By changing, the diameter of the diaphragm becomes variable as shown in FIGS. 9A, 9B, and 9C.
The feature is that the aperture diameter can be changed freely according to the rotation angle.
Although eight diaphragm blades are used in this example, the diaphragm can be made closer to a circle when the number is larger than when the number is small.

The characteristic configuration, function, and effect of the optical system according to the present embodiment have been described above.
Hereinafter, the configuration and functions of other components such as the image sensor and the image processing unit will be described.

For example, as shown in FIG. 4, the image pickup element 220 includes a glass parallel plane plate (cover glass) 221 and an image pickup surface 222 of an image pickup element made up of a CCD or CMOS sensor in order from the fifth lens 216 side. Has been.
Light from the subject OBJ via the imaging optical system 210 </ b> A is imaged on the imaging surface 222 of the imaging element 220.
Note that the subject dispersion image picked up by the image pickup device 220 is an image in which the light wavefront modulation surface 213a is not focused on the image pickup device 220 and a deep light beam and a blurred portion are formed.
And in this embodiment, it is comprised so that the resolution of the distance between objects can be complemented by adding a filter process in the image processing apparatus 240. FIG.
The optical system 210A will be described in detail later.

As shown in FIG. 3, the image sensor 220 forms an image captured by the optical system 210, and forms an image primary image information as a primary image signal FIM of an electrical signal via the analog front end unit 230. It consists of a CCD or CMOS sensor output to the image processing device 240.
In FIG. 3, the imaging element 220 is described as a CCD as an example.

The analog front end unit 230 includes a timing generator 231 and an analog / digital (A / D) converter 232.
The timing generator 231 generates the drive timing of the CCD of the image sensor 220, and the A / D converter 232 converts an analog signal input from the CCD into a digital signal and outputs it to the image processing device 240.

An image processing apparatus (two-dimensional convolution means) 240 constituting a part of the signal processing unit inputs a digital signal of a captured image coming from the previous AFE 230, performs two-dimensional convolution processing, and performs subsequent camera signal processing. Part (DSP) 250.
Filter processing is performed on the optical transfer function (OTF) according to the exposure information of the image processing device 240 and the control device 290. Note that aperture information is included as exposure information.
The image processing device 240 performs a filtering process so as to improve the response of the optical transfer function (OTF) to a plurality of images by the image sensor 220 and eliminate the change of the optical transfer function (OTF) according to the object distance. It has a function of performing (for example, convolution filter processing) and obtains a deep depth of field while depending on a plurality of object distances. Further, the image processing apparatus 240 has a function of performing noise reduction filtering in the first step.
The image processing device 240 has a function of performing processing for improving the contrast by performing filter processing on the optical transfer function (OTF).
The processing of the image processing device 240 will be described in further detail later.

  The camera signal processing unit (DSP) 250 performs processing such as color interpolation, white balance, YCbCr conversion processing, compression, and filing, and stores in the memory 260 and displays images on the image monitoring device 270.

  The control device 290 performs exposure control and has operation inputs from the operation unit 280 and the like, determines the operation of the entire system in accordance with those inputs, and controls the AFE 230, the image processing device 240, the DSP 250, the aperture 213, and the like. It governs mediation control for the entire system.

  Hereinafter, the configuration and function of the optical system and the image processing apparatus according to the present embodiment will be specifically described.

Next, filter processing of the image processing apparatus 240 will be described.
In the present embodiment, the light beam converged by the optical system 210 is regularly dispersed. By inserting the light wavefront modulation element in this way, an image that does not fit anywhere on the imaging element 220 is realized.
In other words, the optical system 210 forms a deep luminous flux (which plays a central role in image formation) and a flare (blurred portion).
As described above, means for restoring the regularly dispersed image to a focused image without moving the optical system 210 by digital processing is a wavefront aberration control optical system system or a depth extension optical system system (DEOS). : Depth Expansion Optical System), and this processing is performed in the image processing apparatus 240.

Here, the basic principle of DEOS will be described.
As shown in FIG. 10, when the subject image f enters the DEOS optical system H, a g image is generated.
This is expressed by the following equation.

(Equation 1)
g = H * f
However, * represents convolution.

  In order to obtain the subject from the generated image, the following processing is required.

(Equation 2)
f = H ^-1 * g

Here, the kernel size and calculation coefficient regarding H will be described.
Let the zoom positions be ZPn, ZPn-1,. In addition, each H function is defined as Hn, Hn-1,.
Since each spot image is different, each H function is as follows.

The difference in the number of rows and / or the number of columns in this matrix is the kernel size, and each number is the operation coefficient.
Here, each H function may be stored in a memory, and the PSF is set as a function of the object distance, and is calculated based on the object distance. By calculating the H function, an optimum object distance is obtained. It may be possible to set so as to create a filter. Alternatively, the H function may be directly obtained from the object distance using the H function as a function of the object distance.

  In this embodiment, as shown in FIG. 3, an image from the optical system 210 is received by the image sensor 220 and input to the image processing device 240 when the aperture is opened, and a conversion coefficient corresponding to the optical system is acquired. The image signal having no dispersion is generated from the dispersion image signal from the image sensor 220 with the acquired conversion coefficient.

  In the present embodiment, DEOS can be employed to obtain high-definition image quality, and the optical system can be simplified and the cost can be reduced.

  As described above, the image processing apparatus 240 receives the primary image FIM from the image sensor 220, and performs a process of extending the depth of field by a convolution process using a filter to form a high-definition final image FNLIM. .

The MTF correction processing of the image processing apparatus 140 is performed after edge enhancement, chroma enhancement, etc., using the MTF of the primary image, which is essentially a low value, as shown by a curve A in FIG. In the process, correction is performed so as to approach (reach) the characteristic indicated by the curve B in FIG.
A characteristic indicated by a curve B in FIG. 11 is a characteristic obtained when the wavefront is not deformed without using the wavefront forming optical element as in the present embodiment, for example.
It should be noted that all corrections in the present embodiment are based on spatial frequency parameters.

In the present embodiment, as shown in FIG. 11, in order to achieve the MTF characteristic curve B that is finally realized with respect to the optically obtained MTF characteristic curve A with respect to the spatial frequency, each spatial frequency is changed to each spatial frequency. On the other hand, the original image (primary image) is corrected by applying strength such as edge enhancement as shown in FIG.
For example, in the case of the MTF characteristic of FIG. 11, the curve of edge enhancement with respect to the spatial frequency is as shown in FIG.

  That is, a desired MTF characteristic curve B is virtually realized by performing correction by weakening edge enhancement on the low frequency side and high frequency side within a predetermined spatial frequency band and strengthening edge enhancement in the intermediate frequency region. To do.

As described above, the imaging apparatus 200 according to the embodiment basically includes the optical system 210 and the imaging element 220 that form a primary image, and the image processing apparatus 140 that forms the primary image into a high-definition final image. In the optical system, a wavefront shaping optical element is newly provided, or an optical element such as glass, plastic or the like is formed for wavefront shaping, thereby forming an imaging wavefront. The image is deformed (modulated), and such a wavefront is imaged on the imaging surface (light-receiving surface) of the imaging device 220 including a CCD or CMOS sensor, and a high-definition image is obtained from the imaged primary image through the image processing device 240. An image forming system.
In the present embodiment, the primary image from the image sensor 220 has a light flux condition with a very deep depth. For this reason, the MTF of the primary image is essentially a low value, and the MTF is corrected by the image processing device 240.

Here, the imaging process in the imaging apparatus 200 in the present embodiment will be considered in terms of wave optics.
A spherical wave diverging from one of the object points becomes a convergent wave after passing through the imaging optical system. At that time, aberration occurs if the imaging optical system is not an ideal optical system. The wavefront is not a spherical surface but a complicated shape. Wavefront optics lies between geometric optics and wave optics, which is convenient when dealing with wavefront phenomena.
When dealing with the wave optical MTF on the imaging plane, the wavefront information at the exit pupil position of the imaging optical system is important.
The MTF is calculated by Fourier transform of the wave optical intensity distribution at the imaging point. The wave optical intensity distribution is obtained by squaring the wave optical amplitude distribution, and the wave optical amplitude distribution is obtained from the Fourier transform of the pupil function in the exit pupil.
Further, since the pupil function is exactly from the wavefront information (wavefront aberration) at the exit pupil position itself, if the wavefront aberration can be strictly numerically calculated through the optical system 210, the MTF can be calculated.

Accordingly, if the wavefront information at the exit pupil position is modified by a predetermined method, the MTF value on the imaging plane can be arbitrarily changed.
In this embodiment, the wavefront shape is mainly changed by the wavefront forming optical element, but the target wavefront is formed by increasing or decreasing the phase (phase, optical path length along the light beam). .
Then, if the target wavefront is formed, the exiting light flux from the exit pupil is formed from a dense portion and a sparse portion of the light, as can be seen from the geometric optical spot image.
The MTF in the luminous flux state shows a low value at a low spatial frequency and a characteristic that the resolving power is managed up to a high spatial frequency.
That is, if this MTF value is low (or such a spot image state in terms of geometrical optics), the phenomenon of aliasing will not occur.
That is, a low-pass filter is not necessary.
Then, the flare-like image that causes the MTF value to be lowered by the image processing device 240 may be removed. Thereby, the MTF value is significantly improved.

  Next, the response of the MTF of this embodiment and the conventional optical system will be considered.

FIG. 13 is a diagram showing the MTF response when the object is at the focal position and when the object is out of the focal position in the case of the conventional optical system.
FIG. 14 is a diagram showing the response of the MTF when the object is at the focal position and when the object is out of the focal position in the optical system of the present embodiment having the light wavefront modulation element.
FIG. 15 is a diagram illustrating a response of the MTF after data restoration of the imaging apparatus according to the present embodiment.

As can be seen from the figure, in the case of an optical system having a light wavefront modulation element, the change in the response of the MTF is less than that in an optical system in which no light wavefront modulation element is inserted even when the object deviates from the focal position.
The response of the MTF is improved by processing the image formed by this optical system using a convolution filter.

The absolute value (MTF) of the OTF of the optical system having the phase plate shown in FIG. 14 is preferably 0.1 or more at the Nyquist frequency.
This is because, in order to achieve the OTF after restoration shown in FIG. 15, the gain is increased by the restoration filter, but the noise of the sensor is also raised at the same time. For this reason, it is preferable to perform restoration without increasing the gain as much as possible at high frequencies near the Nyquist frequency.
In the case of a normal optical system, resolution is achieved if the MTF at the Nyquist frequency is 0.1 or more.
Therefore, if the MTF before restoration is 0.1 or more, the restoration filter does not need to increase the gain at the Nyquist frequency. If the MTF before restoration is less than 0.1, the restored image becomes an image greatly affected by noise, which is not preferable.

  The configuration and processing of the image processing apparatus 240 will be described.

  As shown in FIG. 3, the image processing apparatus 240 includes a raw (RAW) buffer memory 241, a two-dimensional convolution operation unit 242, a kernel data storage ROM 243 as a storage unit, and a convolution control unit 244.

  The convolution control unit 244 controls the convolution process on / off, the screen size, the replacement of kernel data, and the like, and is controlled by the control device 290.

The kernel data storage ROM 243 stores kernel data for convolution calculated from the point image intensity distribution (PSF) of each optical system prepared in advance as shown in FIG. 16, FIG. 17, or FIG. The exposure information determined at the time of setting the exposure is acquired by the control device 290, and the kernel data is selected and controlled through the convolution control unit 244.
The exposure information includes aperture information.

  In the example of FIG. 16, the kernel data A is data corresponding to the optical magnification (× 1.5), the kernel data B is data corresponding to the optical magnification (× 5), and the kernel data C is data corresponding to the optical magnification (× 10).

  In the example of FIG. 17, the kernel data A is data corresponding to the F number (2.8) as aperture information, and the kernel data B is data corresponding to the F number (4).

  In the example of FIG. 18, the kernel data A is data corresponding to an object distance information of 100 mm, the kernel data B is data corresponding to an object distance of 500 mm, and the kernel data C is data corresponding to an object distance of 4 m.

FIG. 19 is a flowchart of a switching process based on exposure information (including aperture information) of the control device 290.
First, exposure information (RP) is detected and supplied to the convolution control unit 244 (ST101).
In the convolution control unit 244, the kernel size and numerical performance coefficient are set in the register from the exposure information RP (ST102).
The image data captured by the image sensor 220 and input to the two-dimensional convolution operation unit 242 via the AFE 230 is subjected to a convolution operation based on the data stored in the register, and is calculated and converted. The transferred data is transferred to the camera signal processing unit 250 (ST103).

  A more specific example of the signal processing unit and kernel data storage ROM of the image processing apparatus 240 will be described below.

FIG. 20 is a diagram illustrating a first configuration example of the signal processing unit and the kernel data storage ROM. For simplification, AFE and the like are omitted.
The example of FIG. 20 is a block diagram when a filter kernel corresponding to exposure information is prepared in advance.

  The image processing apparatus 240 acquires exposure information determined at the time of setting the exposure from the exposure information detection unit 253, and selects and controls kernel data through the convolution control unit 244. The two-dimensional convolution operation unit 242 performs convolution processing using kernel data.

FIG. 21 is a diagram illustrating a second configuration example of the signal processing unit and the kernel data storage ROM. For simplification, AFE and the like are omitted.
The example of FIG. 21 is a block diagram in the case where the image processing device 240 has a noise reduction filter processing step at the beginning, and noise reduction filter processing ST1 corresponding to exposure information is prepared in advance as filter kernel data.

Exposure information determined at the time of exposure setting is acquired from the exposure information detection unit 253, and kernel data is selected and controlled through the convolution control unit 244.
In the two-dimensional convolution operation unit 242, after performing the noise reduction filter processing 1ST1, the color space is converted by the color conversion processing ST2, and then the convolution processing (OTF restoration filter processing) ST3 is performed using the kernel data.
The noise reduction filter process 2ST4 is performed again, and the original color space is restored by the color conversion process ST5. The color conversion process includes, for example, YCbCr conversion, but other conversions may be used.
Note that the second noise processing ST4 can be omitted.

FIG. 22 is a diagram illustrating a third configuration example of the signal processing unit and the kernel data storage ROM. For simplification, AFE and the like are omitted.
The example of FIG. 22 is a block diagram when an OTF restoration filter corresponding to exposure information is prepared in advance.

Exposure information determined at the time of exposure setting is acquired from the exposure information detection unit 253, and kernel data is selected and controlled through the convolution control unit 244.
The two-dimensional convolution calculation unit 242 performs the convolution process ST13 using the OTF restoration filter after the noise reduction filter process 1ST11 and the color conversion process ST12.
The noise reduction filter process 2ST14 is performed again, and the original color space is restored by the color conversion process ST15. The color conversion process includes, for example, YCbCr conversion, but other conversions may be used.
Note that only one of the noise reduction filter processes ST11 and ST14 may be used.

FIG. 23 is a diagram illustrating a fourth configuration example of the signal processing unit and the kernel data storage ROM. For simplification, AFE and the like are omitted.
The example of FIG. 23 is a block diagram in the case where a noise reduction filter processing step is included and a noise reduction filter corresponding to exposure information is prepared in advance as filter kernel data.
Note that the second noise processing ST4 can be omitted.
Exposure information determined at the time of exposure setting is acquired from the exposure information detection unit 253, and kernel data is selected and controlled through the convolution control unit 244.
In the two-dimensional convolution operation unit 242, after performing the noise reduction filter processing 1ST21, the color space is converted by the color conversion processing ST22, and then the convolution processing ST23 is performed using the kernel data.
The noise reduction filter process 2ST24 corresponding to the exposure information is performed again, and the original color space is restored by the color conversion process ST25. The color conversion process includes, for example, YCbCr conversion, but other conversions may be used.
The noise reduction filter process ST21 can be omitted.

  The example in which the filter processing is performed in the two-dimensional convolution calculation unit 242 according to only the exposure information has been described above. However, for example, subject distance information, zoom information, or a calculation coefficient suitable by combining shooting mode information and exposure information Can be extracted or calculated.

  FIG. 24 is a diagram illustrating a configuration example of an image processing apparatus that combines subject distance information and exposure information.

  The imaging apparatus 200A includes a convolution apparatus 401, a kernel / numerical calculation coefficient storage register 402, and an image processing calculation processor 403 as illustrated in FIG.

In this imaging apparatus 200A, the image processing arithmetic processor 403 that has obtained the information about the approximate object distance of the subject and the exposure information read from the object approximate distance information detection apparatus 500 performs an appropriate calculation on the object separation position. The kernel size to be used and its calculation coefficient are stored in the kernel and numerical calculation coefficient storage register 402, and an appropriate calculation is performed by the convolution device 401 that calculates using the value, thereby restoring the image.
In this example, the distance to the main subject is detected by the object approximate distance information detection device 500 including the distance detection sensor, and different image correction processing is performed according to the detected distance.

The above image processing is performed by convolution calculation. To realize this, for example, one type of convolution calculation coefficient is stored in common, and a correction coefficient is stored in advance according to the focal length, The correction coefficient is used to correct the calculation coefficient, and an appropriate convolution calculation can be performed using the corrected calculation coefficient.
In addition to this configuration, the following configuration can be employed.

  The kernel size and the convolution calculation coefficient itself are stored in advance according to the focal length, the convolution calculation is performed using the stored kernel size and calculation coefficient, and the calculation coefficient according to the focal length is stored in advance as a function. In addition, it is possible to employ a configuration in which a calculation coefficient is obtained from this function based on the focal length and a convolution calculation is performed using the calculated calculation coefficient.

  Corresponding to the configuration of FIG. 24, the following configuration can be taken.

At least two or more conversion coefficients corresponding to the aberration caused by the resin lens corresponding to the phase plate are stored in advance in the register 402 as the conversion coefficient storage means according to the subject distance. The image processing arithmetic processor 403 functions as a coefficient selection unit that selects a conversion coefficient according to the distance from the register 402 to the subject based on information generated by the object approximate distance information detection device 500 serving as a subject distance information generation unit. .
Then, the convolution device 401 as the conversion unit converts the image signal using the conversion coefficient selected by the image processing arithmetic processor 403 as the coefficient selection unit.

Alternatively, as described above, the image processing calculation processor 403 as the conversion coefficient calculation unit calculates the conversion coefficient based on the information generated by the object approximate distance information detection device 500 as the subject distance information generation unit, and stores it in the register 402. Store.
Then, the convolution device 401 as the conversion unit converts the image signal by the conversion coefficient obtained by the image processing calculation processor 403 as the conversion coefficient calculation unit and stored in the register 402.

Alternatively, at least one correction value corresponding to the zoom position or zoom amount of the zoom optical system 210 is stored in advance in the register 402 serving as a correction value storage unit. This correction value includes the kernel size of the subject aberration image.
Then, based on the distance information generated by the object approximate distance information detection device 500 serving as the subject distance information generating unit, the image processing arithmetic processor 403 serving as the correction value selecting unit transmits information from the register 402 serving as the correction value storing unit to the subject. Select a correction value according to the distance.
The convolution device 401 as the conversion unit performs image processing based on the conversion coefficient obtained from the register 402 as the second conversion coefficient storage unit and the correction value selected by the image processing arithmetic processor 403 as the correction value selection unit. Perform signal conversion.

As described above, according to the present embodiment, the shape of the modulation surface 213a of the optical wavefront modulation element 213A and the shape of the stop 214 are rotationally symmetric with respect to the optical axis, and the optical wavefront modulation function is manifested. Since the diaphragm 214 is provided with an adjustment function so that it can be switched to non-expression or light wavefront modulation characteristics, the optical system can be simplified and the cost can be reduced. In addition, it is possible to obtain a good restored image with appropriate image quality corresponding to the shooting conditions.
Thereby, optimization can be performed in a situation where contrast is required and a situation where a general image is required.

In addition, the kernel size used in the convolution calculation and the coefficient used in the numerical calculation are made variable, know by the input of the operation unit 280 shown in FIG. There is an advantage that the lens can be designed without worrying about the defocus range and that the image can be restored by convolution with high accuracy.
Further, there is an advantage that a natural image can be obtained without requiring an optical lens that is difficult, expensive, and large in size, and without driving the lens.
The imaging apparatus 200 according to the present embodiment can be used in a DEOS optical system in consideration of the small size, light weight, and cost of consumer devices such as digital cameras and camcorders.
Further, the configuration of the optical system 210 can be simplified, manufacturing becomes easy, and cost reduction can be achieved.

By the way, when a CCD or CMOS sensor is used as an image sensor, there is a resolution limit determined by the pixel pitch, and if the resolution of the optical system exceeds the limit resolution, a phenomenon such as aliasing occurs, which adversely affects the final image. It is a well-known fact that
In order to improve image quality, it is desirable to increase the contrast as much as possible, but this requires a high-performance lens system.

However, as described above, aliasing occurs when a CCD or CMOS sensor is used as an image sensor.
Currently, in order to avoid the occurrence of aliasing, the imaging lens apparatus uses a low-pass filter made of a uniaxial crystal system to avoid the occurrence of aliasing.
The use of a low-pass filter in this way is correct in principle, but the low-pass filter itself is made of crystal, so it is expensive and difficult to manage. Moreover, there is a disadvantage that the use of the optical system makes the optical system more complicated.

As described above, in order to form a high-definition image, the optical system must be complicated in the conventional imaging lens apparatus in spite of the demand for higher-definition image due to the trend of the times. . If it is complicated, manufacturing becomes difficult, and if an expensive low-pass filter is used, the cost increases.
However, according to this embodiment, the occurrence of aliasing can be avoided without using a low-pass filter, and high-definition image quality can be obtained.

  Also, the kernel data storage ROMs of FIGS. 16, 17, and 18 are not necessarily used for the values of optical magnification, F number, size of each kernel, and object distance. Also, the number of kernel data to be prepared is not limited to three.

  In the above description, the information code reading device is described as an example of an electronic device to which the imaging device of the present invention can be applied. However, the imaging device of the present invention is also applicable to a portable electronic device such as a mobile phone and a PDA. Applicable.

1 is an external view showing an example of an information code reading device according to an embodiment of the present invention. It is a figure which shows an example of an information code. It is a block which shows the structural example of the imaging device applied to the information code reader of FIG. It is a figure which shows the basic structural example of the imaging lens unit which forms the optical system which concerns on this embodiment, Comprising: In the optical system which concerns on this embodiment, it adjoined the optical wavefront modulation element (phase modulation element) and aperture_diaphragm | restriction in the whole lens. It is a figure which shows the structural example which implement | achieves the light wavefront modulation in a surface. It is a figure which shows typically the relationship between the optical wavefront modulation element concerning this embodiment, and an aperture_diaphragm | restriction. It is a figure which shows MTF when a light ray passes 1st phase surface PS1 and 2nd phase surface PS2 of FIG. It is a figure which shows light intensity distribution when a light ray passes 1st phase surface PS1 and 2nd phase surface PS2 of FIG. It is a figure which shows the example of a switching aperture_diaphragm | restriction. It is a figure which shows the example of a variable aperture stop. It is a figure for demonstrating the principle of DEOS. It is a figure for demonstrating the MTF correction process in the image processing apparatus which concerns on this embodiment. It is a figure for demonstrating concretely the MTF correction process in the image processing apparatus which concerns on this embodiment. It is a figure which shows the response (response) of MTF when an object exists in a focus position in the case of the conventional optical system, and when it remove | deviated from the focus position. It is a figure which shows the response of MTF when an object exists in a focus position in the case of the optical system of this embodiment which has a light wavefront modulation element, and remove | deviates from a focus position. It is a figure which shows the response of MTF after the data restoration of the imaging device which concerns on this embodiment. It is a figure which shows an example (optical magnification) of the storage data of kernel data ROM. It is a figure which shows the other example (F number) of the storage data of kernel data ROM. It is a figure which shows the other example (object distance) of the storage data of kernel data ROM. It is a flowchart which shows the outline | summary of the optical system setting process of an exposure control apparatus. It is a figure which shows the 1st structural example about a signal processing part and a kernel data storage ROM. It is a figure which shows the 2nd structural example about a signal processing part and a kernel data storage ROM. It is a figure which shows the 3rd structural example about a signal processing part and kernel data storage ROM. It is a figure which shows the 4th structural example about a signal processing part and kernel data storage ROM. It is a figure which shows the structural example of the image processing apparatus which combines subject distance information and exposure information. It is a figure which shows typically the structure and light beam state of a general imaging lens apparatus. FIG. 26A is a diagram showing a spot image on the light receiving surface of the imaging element of the imaging lens device of FIG. 25, where FIG. 25A shows a case where the focal point is shifted by 0.2 mm (Defocus = 0.2 mm), and FIG. In the case (Best focus), (C) is a diagram showing each spot image when the focal point is shifted by -0.2 mm (Defocus = -0.2 mm).

Explanation of symbols

  DESCRIPTION OF SYMBOLS 200 ... Imaging device, 210 ... Optical system, 211 ... 1st lens, 212 ... 2nd lens, 213 ... 3rd lens, 214 ... Aperture, 215 ... 4th Lens 220 ... Imaging device PS1 ... First phase plane PS2 ... Second phase plane 230 ... Analog front end (AFE) 240 ... Image processor 250 ... Camera signal processing unit, 280 ... operation unit, 290 ... control device, 242 ... two-dimensional convolution operation unit, 243 ... kernel data ROM, 244 ... convolution control unit.

Claims (7)

An optical system including a lens and a light wavefront modulation element;
An image sensor that images a subject image that has passed through the optical system;
Limiting the luminous flux passing through the light wavefront modulation element, and having a diaphragm capable of changing the restricted portion,
An imaging apparatus that switches on or off of an optical wavefront modulation function or switching optical wavefront modulation characteristics by changing the diaphragm. The aperture is
The imaging apparatus according to claim 1, comprising a plurality of types of apertures in which the restricted portion is different. The aperture is
The imaging device according to claim 1, further comprising a variable diaphragm that makes the restricted portion variable. An image processing device that performs predetermined image processing on an image signal obtained from the image sensor;
The image processing apparatus includes:
The imaging device according to any one of claims 1 to 3, wherein the presence or absence of image processing or the content of image processing is switched according to the state of the aperture. The imaging apparatus according to claim 1, wherein the optical wavefront modulation element is disposed in the vicinity of the diaphragm. The imaging device according to any one of claims 1 to 5, wherein a light wavefront modulation function of the light wavefront modulation element and a shape of the diaphragm are substantially rotationally symmetric with respect to an optical axis. An electronic device having an imaging device,
The imaging device
An optical system including a lens and a light wavefront modulation element;
An image sensor that images a subject image that has passed through the optical system;
Limiting the luminous flux passing through the light wavefront modulation element, and having a diaphragm capable of changing the restricted portion,
An electronic device that switches on or off the light wavefront modulation function or switching the light wavefront modulation characteristics by changing the aperture.

Images