JP5804105B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

  There is known a focus detection device that calculates a relative displacement amount (image shift amount, shift amount) between a pair of data strings corresponding to a pair of relatively distorted images (see, for example, Patent Document 1). ). In this apparatus, for a pair of data strings, the correlation amount is calculated while changing the shift amount k using a correlation calculation formula including multiplication of data between the pair of data strings.

JP 2007-333720 A

  However, the correlation calculation formula for calculating the correlation amount in the conventional focus detection apparatus described above has inappropriate calculation characteristics depending on the state of the image data (degree of vignetting, degree of noise, etc.), and when applied uniformly, the image There is a problem that the displacement detection accuracy is lowered.

(1) According to the first aspect of the present invention, the first photoelectric conversion unit that receives one light beam and the second photoelectric conversion unit that receives the other light beam out of a pair of light beams that have passed through different pupil regions of the photographing optical system. An image sensor having a plurality of provided pixels, a first data string from the plurality of first photoelectric conversion units, and a second data string from the plurality of second photoelectric conversion units are changed relative to each other while changing a displacement amount. A correlation calculation unit that calculates a plurality of correlation amounts between the first data string and the second data string by a plurality of correlation calculation expressions, and calculates an addition correlation amount obtained by adding the plurality of correlation amounts; A lens drive control unit that converts the added correlation amount into a defocus amount and generates a control signal for driving the imaging optical system based on the defocus amount.
(2) According to the invention of claim 2, in the imaging device according to claim 1, at least one of the plurality of correlation calculation expressions includes one data in the first data string and the second It is a correlation calculation expression including multiplication with one data in a data string.
(3) The invention of claim 3 is the imaging apparatus according to claim 2, wherein there are a plurality of correlation calculation expressions including the multiplication, and the correlation calculation expression including the multiple multiplications is included in the first data string. Of one of the data and one data in the second data string, and another data in the first data string and another data in the second data string A partial correlation calculation expression including multiplication, an interval between one data in the first data string and another data in the first data string, and 1 in the second data string The interval between one piece of data and the other one piece of data in the second data string is a predetermined interval.
(4) According to a fourth aspect of the present invention, in the imaging device according to the third aspect, the correlation calculation formula including the plurality of multiplications includes two types of the partial correlation calculation formulas having different predetermined intervals. And
(5) According to the invention of claim 5, the first photoelectric conversion unit that receives one light beam and the second photoelectric conversion unit that receives the other light beam among a pair of light beams that have passed through different pupil regions of the photographing optical system. An image sensor having a plurality of provided pixels, a first data string from the plurality of first photoelectric conversion units, and a second data string from the plurality of second photoelectric conversion units are changed relative to each other while changing a displacement amount. A correlation calculation unit that calculates a plurality of correlation amounts by calculating a plurality of correlation amounts between the first data string and the second data string by a plurality of correlation calculation formulas, and the plurality of correlation calculation formulas A control signal that determines a defocus amount with high reliability according to the correlation parameter obtained by the above, and drives the imaging optical system based on an average defocus amount of the defocus amounts determined to have high reliability a body drive control unit for generating a Characterized in that it.
(6) The invention of claim 6 is the imaging apparatus according to claim 5, wherein the correlation parameter is at least one of a minimum value of correlation values and a value proportional to contrast .
(7) The invention of claim 7 is the imaging device according to claim 5 or 6, wherein the plurality of correlation calculation expressions have different characteristics with respect to distortion between the first data string and the second data string. It is an arithmetic expression.
(8) The invention of claim 8 is the imaging device according to claim 7, wherein when the distortion is large, one of the first data strings is determined as the defocus amount determined to have high reliability. A correlation amount between the first data string and the second data string calculated by a correlation calculation expression including multiplication of data and one data in the second data string is extracted.
(9) The invention according to claim 9 is the imaging apparatus according to claim 5 or 6, wherein the plurality of correlation calculation formulas have different characteristics with respect to noise applied to the first data string and the second data string. It is a formula.
(10) The invention of claim 10 is the imaging device according to claim 9, wherein when the noise is large, one of the first data strings is determined as the defocus amount determined to be high in reliability. A correlation amount between the first data string and the second data string calculated by a correlation calculation expression including multiplication of data and one data in the second data string is extracted.
(11) The invention according to claim 11 is the imaging device according to claim 5 or 6, wherein the plurality of correlation calculation formulas have different characteristics with respect to spatial frequencies of the first data string and the second data string. It is a formula.
(12) According to a twelfth aspect of the present invention, in the imaging apparatus according to any one of the first to eleventh aspects of the present invention, the imaging apparatus includes a calculation unit that calculates the displacement amount from which the extreme value of the correlation amount is obtained. To do.

  According to the present invention, the correlation amount of a pair of data strings can be accurately calculated for a pair of data strings in various situations.

Cross-sectional view of the camera showing the configuration of the camera of one embodiment The figure which shows the focus detection position on the photographing screen of the interchangeable lens Front view showing detailed configuration of image sensor Front view showing configuration of imaging pixel Front view showing configuration of focus detection pixel Diagram showing spectral characteristics of imaging pixels Diagram showing spectral characteristics of focus detection pixels Cross section of imaging pixel Cross section of focus detection pixel The figure which shows the structure of the focus detection optical system of the pupil division type phase difference detection method using a micro lens The flowchart which shows the imaging operation of the digital still camera (imaging device) of one embodiment The flowchart which shows the focus detection calculation process of one embodiment A diagram for explaining vignetting of a focus detection light beam Figure when viewing the distance measuring pupil plane in the direction of the optical axis from the planned focal plane 13 and 14, a pair of images received by the focus detection pixel row near the position x0 (image height 0) and a pair of images received by the focus detection pixel row near the position x1 (image height h). Of the intensity distribution (vertical axis is light intensity, horizontal axis is the position on the shooting screen) Graph of a pair of image data FIG. 16A is a graph when the correlation amounts C1 (k), C2 (k), and C (k) are calculated in an image shift state in which the pair of image data shown in FIG. FIG. 16B is a graph when the correlation amounts C1 (k), C2 (k), and C (k) are calculated in an image shift state in which the pair of image data shown in FIG. FIG. 16C is a graph when the correlation amounts C1 (k), C2 (k), and C (k) are calculated in an image shift state in which the pair of image data shown in FIG. When the correlation amounts C1 (k), C2 (k), and C (k) are calculated in an image shift state in which the pair of image data shown in FIG. 16 is relatively shifted from −1 to +1 of the data pitch, the correlation is calculated. Graph of calculation results of image displacement detection from the extreme value of quantity (C1 (k) is indicated by ■, C2 (k) is indicated by □, and C (k) is indicated by ×) in the figure) The figure for explaining the evaluation method of the focus detection result Diagram comparing the characteristics of correlation equations A, B, and C Diagram showing a pair of image data FIG. 23 shows the calculation result when image shift detection is performed by applying the correlation calculation formulas A, B, and C to the image data in which the pair of image data in FIG. 23 is relatively shifted from the data pitch −1 to the data pitch +1. Illustration The figure which shows the difference | error of the actual image shift amount shown in FIG. 24, and the image shift amount of a calculation result Graph of correlation amount C (k) when correlation calculation formula A is applied to image data in which the pair of image data in FIG. Graph of correlation amount C (k) when correlation calculation formula B is applied to image data in which the pair of image data in FIG. Graph of correlation amount C (k) when correlation calculation formula C is applied to image data in which the pair of image data in FIG. 23 is relatively shifted by data pitch −1. The figure which compared the relevance of spn and the frequency component of data in correlation formula B, C Diagram showing a pair of image data When the image shift detection is performed by applying the correlation formula B (spn = 1, 10) to the image data in which the pair of image data shown in FIG. 30 is relatively shifted from −1 to +1 of the data pitch. Figure showing calculation results Flow chart showing focus detection calculation processing Flow chart showing focus detection calculation processing Flow chart showing focus detection calculation processing The figure which shows the image pick-up element of a modification The front view of the focus detection pixel used with the image pick-up element of the modification shown in FIG. The figure for demonstrating the focus detection operation | movement of a pupil division system by the focus detection pixel of the image pick-up element shown in FIG. The figure for demonstrating the focus detection operation | movement of a re-imaging pupil division system

  As an imaging apparatus equipped with the focus detection apparatus of one embodiment, a lens interchangeable digital still camera will be described as an example. FIG. 1 is a cross-sectional view of a camera showing the configuration of the camera of one embodiment. A digital still camera 201 according to an embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 via a mount unit 204. An interchangeable lens 202 having various imaging optical systems can be attached to the camera body 203 via a mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, an aperture 211, a lens drive control device 206, and the like. The lens drive control device 206 includes a microcomputer (not shown), a memory, a drive control circuit, and the like. The lens drive control device 206 includes drive control for adjusting the focus of the focusing lens 210 and the aperture diameter of the aperture 211, zooming lens 208, and focusing. In addition to detecting the state of the lens 210 and the aperture 211, the lens information is transmitted and the camera information is received through communication with a body drive control device 214 described later. The aperture 211 forms an aperture having a variable aperture diameter at the center of the optical axis in order to adjust the amount of light and the amount of blur.

  The camera body 203 includes an imaging element 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the imaging element 212, imaging pixels are two-dimensionally arranged, and focus detection pixels are incorporated in portions corresponding to focus detection positions. Details of the image sensor 212 will be described later.

  The body drive control device 214 includes a microcomputer, a memory, a drive control circuit, and the like, and controls the drive of the image sensor 212, reads out the image signal and the focus detection signal, performs the focus detection calculation based on the focus detection signal, and the focus of the interchangeable lens 202. The adjustment is repeated, and image signal processing and recording, camera operation control, and the like are performed. The body drive control device 214 communicates with the lens drive control device 206 via the electrical contact 213 to receive lens information and send camera information (defocus amount, aperture value, etc.).

  The liquid crystal display element 216 functions as an electric view finder (EVF). The liquid crystal display element driving circuit 215 displays a through image by the imaging element 212 on the liquid crystal display element 216, and the photographer can observe the through image through the eyepiece lens 217. The memory card 219 is an image storage that stores an image captured by the image sensor 212.

  A subject image is formed on the light receiving surface of the image sensor 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and an image signal and a focus detection signal are sent to the body drive control device 214.

  The body drive control device 214 calculates the defocus amount based on the focus detection signal from the focus detection pixel of the image sensor 212 and sends the defocus amount to the lens drive control device 206. The body drive control device 214 processes the image signal from the image sensor 212 to generate an image, stores the image in the memory card 219, and sends the through image signal from the image sensor 212 to the liquid crystal display element drive circuit 215. The through image is displayed on the liquid crystal display element 216. Further, the body drive control device 214 sends aperture control information to the lens drive control device 206 to control the aperture of the aperture 211.

  The lens drive controller 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or a lookup prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the table.

  The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens drive amount. Further, the lens drive control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

  FIG. 2 is a diagram showing a focus detection position on the photographing screen of the interchangeable lens 202, and a region (focus detection) in which a focus detection pixel array on the image sensor 212 described later samples an image on the photographing screen when focus detection is performed. An example of an area and a focus detection position is shown. In this example, focus detection areas 101 to 103 are arranged at the center and three locations above and below the rectangular shooting screen 100. Focus detection pixels are linearly arranged in the longitudinal direction of the focus detection area indicated by a rectangle.

  Although details will be described later, in the focus detection areas 101 to 103 shown in FIG. 2, the focus detection areas 102 and 103 around the screen are arranged along the radial direction from the center of the screen, and the focus detection area 101 at the center of the screen. Compared to the above, the focus detection light beam is easily vignetted, and the pair of focus detection signal data sequences detected in the focus detection areas 102 and 103 are relatively distorted, and the identity is lost. However, according to the correlation calculation method of this embodiment, even when relative distortion occurs in such a pair of signal data strings and the identity is lost, the correlation can be accurately calculated. An accurate focus detection result can be obtained.

  FIG. 3 is a front view showing a detailed configuration of the image sensor 212, and shows an enlarged vicinity of the focus detection area 101 on the image sensor 212. Imaging pixels 310 are densely arranged in a two-dimensional square lattice pattern on the imaging element 212, and focus detection pixels 313 and 314 for focus detection are adjacent to each other on a vertical straight line at a position corresponding to the focus detection area 101. Are alternately arranged. Although not shown, the configuration in the vicinity of the focus detection areas 102 and 103 is the same as the configuration shown in FIG.

  As illustrated in FIG. 4, the imaging pixel 310 includes a microlens 10, a photoelectric conversion unit 11, and a color filter (not shown). There are three types of color filters, red (R), green (G), and blue (B), and the respective spectral sensitivities have the characteristics shown in FIG. In the image pickup device 212, image pickup pixels 310 having respective color filters are arranged in a Bayer array.

  As shown in FIG. 5A, the focus detection pixel 313 includes the microlens 10 and the photoelectric conversion unit 13, and the photoelectric conversion unit 13 has a semicircular shape. In addition, the focus detection pixel 314 includes the microlens 10 and the photoelectric conversion unit 14 as illustrated in FIG. 5B, and the photoelectric conversion unit 14 has a semicircular shape. When the focus detection pixel 313 and the focus detection pixel 314 are displayed with the microlens 10 superimposed, the photoelectric conversion units 13 and 14 are arranged in the vertical direction. The focus detection pixels 313 and the focus detection pixels 314 are alternately arranged in the vertical direction (alignment direction of the photoelectric conversion units 13 and 14) in the focus detection areas 101 to 103.

  The focus detection pixels 313 and 314 are not provided with a color filter to increase the amount of light, and the spectral characteristics of the focus detection pixels 313 and 314 include the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). Spectral characteristics (see FIG. 7). That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 6, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, the red pixel, and the blue pixel. ing.

  The focus detection pixels 313 and 314 for focus detection are arranged in a column where B and G of the imaging pixel 310 should be arranged. The focus detection pixels 313 and 314 for focus detection are arranged in a column in which the B and G of the imaging pixel 310 are to be arranged in view of human visual characteristics when an interpolation error occurs in the pixel interpolation processing. This is because the interpolation error of the blue pixel is less noticeable than the interpolation error of the red pixel.

The photoelectric conversion unit 11 of the imaging pixel 310 is designed so as to receive all the light beams that pass through the exit pupil diameter (for example, F1.0) of the brightest interchangeable lens by the microlens 10. The photoelectric conversion units 13 and 14 of the focus detection pixels 313 and 314 are connected to the microlens 10.
Thus, the shape is designed so as to receive all the light beams passing through a predetermined region (for example, F2.8) of the exit pupil of the interchangeable lens.

  FIG. 8 is a cross-sectional view of the imaging pixel 310. In the imaging pixel 310, the microlens 10 is disposed in front of the photoelectric conversion unit 11 for imaging, and the shape of the photoelectric conversion unit 11 is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on the semiconductor circuit substrate 29. A color filter (not shown) is arranged between the microlens 10 and the photoelectric conversion unit 11.

  FIG. 9A is a cross-sectional view of the focus detection pixel 313. In the focus detection pixel 313 disposed in the focus detection area 101 at the center of the screen, the microlens 10 is disposed in front of the photoelectric conversion unit 13, and the shape of the photoelectric conversion unit 13 is projected forward by the microlens 10. The photoelectric conversion unit 13 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by a semiconductor image sensor manufacturing process. Note that the cross-sectional structure of the focus detection pixels 313 arranged in the focus detection areas 102 and 103 at the top and bottom of the screen is the same as the cross-sectional structure shown in FIG.

  FIG. 9B is a cross-sectional view of the focus detection pixel 314. In the focus detection pixel 314 disposed in the focus detection area 101 in the center of the screen, the microlens 10 is disposed in front of the photoelectric conversion unit 14, and the shape of the photoelectric conversion unit 14 is projected forward by the microlens 10. The photoelectric conversion unit 14 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by the manufacturing process of the semiconductor image sensor. Note that the cross-sectional structure of the focus detection pixels 314 arranged in the focus detection areas 102 and 103 at the top and bottom of the screen is the same as the cross-sectional structure shown in FIG.

  FIG. 10 shows a configuration of a pupil division type phase difference detection type focus detection optical system using a microlens. The focus detection pixel portion is shown in an enlarged manner. In the figure, reference numeral 90 denotes an exit pupil set at a distance d forward from the microlens arranged on the planned imaging plane of the interchangeable lens 202 (see FIG. 1). This distance d is a distance determined according to the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and is referred to as a distance measuring pupil distance in this specification. 91 is an optical axis of the interchangeable lens, 10a to 10d are microlenses, 13a, 13b, 14a, and 14b are photoelectric conversion units, 313a, 313b, 314a, and 314b are focus detection pixels, and 73, 74, 83, and 84 are focus detection light fluxes. It is.

  Reference numeral 93 denotes an area of the photoelectric conversion units 13a and 13b projected by the microlenses 10a and 10c, and is referred to as a distance measuring pupil in this specification. In FIG. 10, an elliptical region is shown for easy understanding, but the shape of the photoelectric conversion unit is actually an enlarged projection shape. Similarly, 94 is an area of the photoelectric conversion units 14a and 14b projected by the microlenses 10b and 10d, and is a distance measuring pupil. In FIG. 10, an elliptical region is shown for easy understanding, but the shape of the photoelectric conversion unit is actually an enlarged projection shape.

  In FIG. 10, four focus detection pixels 313a, 313b, 314a, and 314b adjacent to the photographing optical axis are schematically illustrated. However, other focus detection pixels in the focus detection area 101 also have a peripheral portion of the screen. Also in the focus detection pixels in the focus detection areas 102 and 103, the photoelectric conversion units are configured to receive the light beams coming from the corresponding distance measurement pupils 93 and 94 to the respective microlenses. The arrangement direction of the focus detection pixels is made to coincide with the arrangement direction of the pair of distance measuring pupils, that is, the arrangement direction of the pair of photoelectric conversion units.

The microlenses 10a to 10d are disposed in the vicinity of the planned imaging plane of the interchangeable lens 202 (see FIG. 1), and the shapes of the photoelectric conversion units 13a, 13b, 14a, and 14b disposed behind the microlenses 10a to 10d. Is the distance pupil distance d from the microlenses 10a to 10c.
Projected onto the exit pupil 90 separated by a distance, the projection shape forms distance measuring pupils 93 and 94. That is, the relative relationship between the microlens and the photoelectric conversion unit in each focus detection pixel is such that the projection shape (ranging pupils 93 and 94) of the photoelectric conversion unit of each focus detection pixel matches on the exit pupil 90 at the projection distance d. Thus, the projection position of the photoelectric conversion unit in each focus detection pixel is determined.

  The photoelectric conversion unit 13a passes through the distance measuring pupil 93 and outputs a signal corresponding to the intensity of the image formed on the microlens 10a by the light beam 73 directed to the microlens 10a. Similarly, the photoelectric conversion unit 13b outputs a signal corresponding to the intensity of the image formed on the microlens 10c by the light beam 83 passing through the distance measuring pupil 93 and directed to the microlens 10c. Further, the photoelectric conversion unit 14a outputs a signal corresponding to the intensity of the image formed on the microlens 10b by the light beam 74 passing through the distance measuring pupil 94 and directed to the microlens 10b. Similarly, the photoelectric conversion unit 14b outputs a signal corresponding to the intensity of the image formed on the microlens 10d by the light beam 84 passing through the distance measuring pupil 94 and directed to the microlens 10d.

  A large number of the two types of focus detection pixels described above are arranged in a straight line, and the output of the photoelectric conversion unit of each pixel is grouped into a distance measurement pupil 93 and an output group corresponding to the distance measurement pupil 94, thereby measuring the distance measurement pupil 93 and the measurement pupil 93. Information on the intensity distribution of the pair of images formed on the pixel array by the focus detection light beams that respectively pass through the distance pupil 94 is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method. Further, by performing a conversion operation according to the center-of-gravity distance between the pair of distance measuring pupils on the image shift amount, the current imaging plane (the focus determined on the imaging screen 100) with respect to the planned imaging plane (the position of the microlens array). The deviation (defocus amount) of the actual image plane at the detection position is calculated.

  FIG. 11 is a flowchart illustrating an imaging operation of the digital still camera (imaging device) according to the embodiment. Each processing step is executed by the body drive control device 214. When the power of the camera is turned on in step S100, the body drive control device 214 starts the imaging operation after step S110. In step S110, the data of the imaging pixels is read out and displayed on the electronic viewfinder. In subsequent step S120, a pair of image data corresponding to the pair of images is read from the focus detection pixel array. The focus detection area is assumed to be selected in advance by the photographer using one of the focus detection areas 101 to 103 using a focus detection area selection member (not shown).

  In step S130, an image shift detection calculation process (correlation calculation process) described later is performed based on the read pair of image data, and the image shift amount is calculated and converted into a defocus amount. In step S140, it is checked whether or not the focus is close, that is, whether or not the calculated absolute value of the defocus amount is within a predetermined value. If it is determined that the lens is not in focus, the process proceeds to step S150, the defocus amount is transmitted to the lens drive controller 206, and the focusing lens 210 of the interchangeable lens 202 is driven to the focus position. Then, it returns to step S110 and repeats the operation | movement mentioned above.

  Even when focus detection is impossible, the process branches to this step, a scan drive command is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to scan from infinity to the nearest. Then, it returns to step S110 and repeats the operation | movement mentioned above.

  If it is determined in step S140 that the focus is close, the process proceeds to step S160, and it is determined whether or not a shutter release has been performed by operating a shutter button (not shown). If it is determined that the shutter release has not been performed, the process returns to step S110 and the above-described operation is repeated. On the other hand, if it is determined that the shutter release has been performed, the process proceeds to step S170, where an aperture adjustment command is transmitted to the lens drive control unit 206, and the aperture value of the interchangeable lens 202 is set to the control F value (F set by the photographer or automatically). Value). When the aperture control is finished, the image sensor 212 is caused to perform an imaging operation, and image data is read from the image pickup pixel 310 and all the focus detection pixels 313 and 314 of the image pickup element 212.

  In step S180, pixel interpolation of pixel data at each pixel position in the focus detection pixel row is performed based on data of imaging pixels around the focus detection pixel. In the subsequent step S190, image data composed of the imaged pixel data and the interpolated data is stored in the memory card 219, and the process returns to step S110 to repeat the above-described operation.

  FIG. 12 is a flowchart showing details of the focus detection calculation processing in step S130 of FIG. Each processing step is executed by the body drive control device 214. The body drive control device 214 starts this focus detection calculation process (correlation calculation process) from step S200.

In step S210, a high frequency cut filter process as shown in the equation (1) is performed on a pair of data strings (α _{1 to} α _M , β _{1 to} β _M : M is the number of data) output from the focus detection pixel column. The first data string (A _{1 to} A _N ) and the second data string (B _{1 to} B _N ) are generated. As a result, it is possible to remove high-frequency noise components that adversely affect correlation processing and other high-frequency components from the data string. Note that the processing in step S210 can be omitted when the calculation time is shortened or when it is already known that there is little high-frequency component since the focus has been greatly defocused.
A _n = α _n + 2 · α _{n + 1} + α _{n + 2} ,
B _n = β _n + 2 · β _{n + 1} + β _{n + 2} (1)
In the formula (1), n = 1 to N−2.

The data strings A _n and B _n should ideally be the same data string relatively shifted, but a pair of data strings obtained with the focus detection pixels of the above-described pupil division type phase difference detection method. Then, the identity may be lost due to vignetting of the focus detection light beam.

  FIG. 13 is a diagram for explaining vignetting (vignetting) of the focus detection light beam. In FIG. 13, a pair of focus detection pixels at a position x0 (image height 0) and a position x1 (image height h) are respectively a distance measurement pupil region 93, A pair of focus detection light fluxes 53, 54 and 63, 64 passing through 94 is received. When there is an aperture stop 96 of the optical system on the front surface d1 (<d) 95 of the planned focal plane 92, a pair of focus detection light beams 53 received by the pair of focus detection pixels at the position x0 (image height 0). , 54 causes vignetting symmetrically with respect to the optical axis 91 due to the aperture 96, so that the balance of the amount of light received by the pair of focus detection pixels remains unchanged.

  On the other hand, the pair of focus detection light beams 63 and 64 received by the pair of focus detection pixels at the position x1 (image height h) cause vignetting asymmetrically by the aperture 96, and thus a pair of focus detection pixels. The balance of the amount of light received by the camera will be lost.

  FIG. 14 is a diagram when the distance measuring pupil plane 90 is viewed from the planned focal plane 92 in the direction of the optical axis 91. It can be seen that the focus detection light beam 64 has a large vignetting due to the aperture opening 96, whereas the focus detection light beam 63 has less vignetting due to the aperture opening 96.

  FIGS. 15A and 15B show a pair of images received by the focus detection pixel column in the vicinity of the position x0 (image height 0) and the vicinity of the position x1 (image height h) in the states of FIGS. 2 shows the intensity distribution of a pair of images received by the focus detection pixel array (the vertical axis indicates the amount of light, and the horizontal axis indicates the position on the photographing screen). When the vignetting balance of the focus detection light beam is balanced, as shown in FIG. 15A, the pair of image signals 400 and 401 are obtained by simply shifting the same image signal function in the horizontal direction. Yes. On the other hand, when the balance of vignetting of the focus detection light beam is lost (that is, distortion occurs between the pair of image signal data strings), as shown in FIG. The image signals 402 and 403 are not the same signal that is relatively shifted.

The relationship between the pair of image signal data sequences F (x) and G (x) when vignetting (vignetting) occurs in the focus detection light beam is roughly divided into two as follows. First, when the degree of vignetting is small, the pair of image signals has a ratio of a predetermined ratio as shown in the following equation.
F (x) = b0 · G (x) (2)
In the formula (2), b0 is a constant. In this embodiment, when the pair of data strings F (x) and G (x) have the relationship of the expression (2), it is said that “zero-order vignetting” occurs in the pair of data strings. On the other hand, when the degree of vignetting is large, the pair of image signals have a linear function relationship with respect to the position x as in the following equation.
F (x) = (b1 + a1.x) .G (x) (3)
In the formula (3), a1 and b1 (> b0) are constants.

In this embodiment, when the pair of data strings F (x) and G (x) have the relationship of the expression (3), it is said that “primary vignetting” occurs in the pair of data strings. In the state where primary vignetting has occurred, the following equation is adopted as a partial correlation calculation equation capable of detecting image shift. spn is a calculation parameter related to the size of a range including a neighboring data string centered on a pair of attention data, and the data string in the range is used for the correlation calculation processing.
_{E (k) = A i ·} B i + spn + k -B i + k · A i + spn ··· (4)

Returning to FIG. 12, the description will be continued. In step S220, the body drive control device 214 relatively shifts the first data strings A _{1 to An} and the second data strings B _{1 to} B _n (shift amount k), and the first data string and the second data string. A plurality of partial correlation calculation expressions (E1 (k) and E2 (k) when spn = 1 and 10 in the expression (4)) are added, and the absolute values of the calculation results are added, and then the added value Are integrated over a predetermined interval of data.

That is, if the calculation results of a plurality of correlation calculation formulas are partial correlation amounts E1 (k) and E2 (k), the total correlation amount C (k) between a pair of data strings in the shift amount k is ( 7) It is given by the equation.
E1 (k) = A _i · B _{i + 1 + k} −B _{i + k} · A _{i + 1} (5),
E2 (k) = A _i · B _{i + 10 + k} −B _{i + k} · A _{i + 10} (6),
C (k) = Σ (| E1 (k) | + | E2 (k) |) (7)
The shift amount k is an integer, and is a relative shift amount with the detection pitch of a pair of data as a unit. Further, the integration calculation (Σ) in the equation (7) is performed over a predetermined section of the data string.

Here, as shown in equation (7), the absolute values of the calculation results of the two partial correlation calculation equations with different calculation parameters spn are taken and added, and the total value is integrated to add up the total correlation amount. The advantage of obtaining C (k) will be described in comparison with the total correlation amounts C1 (k) and C2 (k) when the absolute values of E1 (k) and E2 (k) are added alone. .
C1 (k) = Σ (| E1 (k) |) (8),
C2 (k) = Σ (| E2 (k) |) (9)

  16 to 20 are simulation data for explaining the characteristics of the correlation amounts C (k), C1 (k), and C2 (k). FIG. 16 is a graph of a pair of image data (represented by ■ and □ in the figure), with the horizontal axis representing the data position and the vertical axis representing the data value. However, in FIG. 16, one of the pair of image data is represented by being shifted by one data position with respect to the other in order to prevent the pair of image data from being overlapped and difficult to understand. 16A shows a case where primary vignetting occurs between a pair of image data (sin waveform: high frequency), and FIG. 16B shows primary vignetting between a pair of image data (sin waveform: medium frequency). FIG. 16C shows a case where primary vignetting occurs between a pair of image data (sin waveform: low frequency). Note that the cycle of the image data in FIG. 16B is 10 times the data pitch, and coincides with the calculation parameter spn = 10 of C2 (k).

  17 to 19 calculate the correlation amounts C1 (k), C2 (k), and C (k) in the image shift state in which the pair of image data shown in FIG. 16 is relatively shifted by the data pitch -1. The horizontal axis represents the shift amount k (unit: data pitch), and the vertical axis represents the values of the correlation amounts C1 (k), C2 (k), and C (k).

  17 (a), 17 (b), and 17 (c) show correlation amounts C1 (k), C2 (k), and C (k) corresponding to the image data (high frequency) shown in FIG. 16 (a). It is a graph of. 18 (a), 18 (b) and 18 (c) show the correlation amounts C1 (k), C2 (k), C corresponding to the image data (medium frequency) shown in FIG. 16 (b). It is a graph of (k). Further, FIGS. 19 (a), 19 (b) and 19 (c) show correlation amounts C1 (k), C2 (k), C corresponding to the image data (low frequency) shown in FIG. 16 (c). It is a graph of (k).

  20A, 20C, and 20E show the correlation amounts C1 (k), C2 in an image shift state in which the pair of image data shown in FIG. 16 is relatively shifted from −1 to +1 of the data pitch. When (k) and C (k) are calculated, a calculation result obtained by detecting an image shift from the extreme value of the correlation amount using a three-point interpolation method (to be described later) ), C2 (k) is indicated by □, and C (k) is indicated by ×. In these figures, the horizontal axis represents the actual image shift amount (unit: data pitch), and the vertical axis represents the image shift amount of the calculation result.

  20 (b), (d), and (f) show errors between the actual image shift amount shown in FIGS. 20 (a), (c), and (e) and the calculated image shift amount (in the figure). Are marked with C1 (k), marked with C2 (k), and marked with C. In these figures, the horizontal axis represents the actual image shift amount (unit: data pitch), and the vertical axis represents the error amount (unit: data pitch).

  FIGS. 20A and 20B are graphs of image shift amounts and errors corresponding to the pair of image data (high frequency) shown in FIG. FIGS. 20C and 20D are graphs of image shift amounts and errors corresponding to the image data (medium frequency) shown in FIG. Further, FIGS. 20E and 20F are graphs of image shift amounts and errors corresponding to the image data (low frequency) shown in FIG.

The characteristics of the correlation amounts C1 (k), C2 (k), and C (k) will be described again with reference to FIGS.
For a pair of image data composed of high-frequency components in which primary vignetting occurs as shown in FIG. 16A, the correlation amount C1 (k) graph shown in FIG. 17A shows a sharp drop in the true image deviation amount. The minimum value is shown, and the error in the calculation result of the image shift amount is small as shown by the ▪ marks in FIGS. 20 (a) and 20 (b). However, as shown in FIG. 17B, the minimum value of the correlation amount C2 (k) is floated, and the calculation result of the image shift amount is shown as indicated by □ in FIGS. 20A and 20B. The error is large. Also, as shown in FIG. 17C, the correlation value C (k) graph has few floats of the minimum value, and the calculation result of the image shift amount as shown by x in FIGS. 20A and 20B. The error is not great.

  Further, for a pair of image data composed of medium frequency components in which primary vignetting occurs as shown in FIG. 16B, the graph of the correlation amount C1 (k) shown in FIG. 18A shows the true image deviation amount. The minimum value of the sharp drop is shown, and there is little error in the calculation result of the image shift amount indicated by the ■ marks in FIGS. 20 (c) and 20 (d). However, in the graph of the correlation amount C2 (k) shown in FIG. 18B, the position of the minimum value is replaced with the position of the maximum value, and the amount of image shift indicated by □ in FIGS. 20C and 20D. The calculation result is completely out of the graph. Further, in the graph of the correlation amount C (k) shown in FIG. 18 (c), the float of the minimum value is small, and the error of the calculation result of the image shift amount shown by x in FIGS. 20 (c) and (d) is also small ( 20 (c) and 20 (d) almost overlap with the result of C1 (k)). When the data period and the calculation parameter spn of the correlation calculation match, the error becomes very large as described above, and the image shift detection result is not reliable.

  Further, for the pair of image data composed of low frequency components in which the first order vignetting shown in FIG. 16C, the graph of the correlation amount C1 (k) shown in FIG. As shown in FIG. 20, the error in the calculation result of the image shift amount indicated by the black square in FIGS. However, the graph of the correlation amount C2 (k) shown in FIG. 19 (b) shows a sharp minimum value with a true image shift amount, and the image shift amount indicated by □ in FIGS. 20 (e) and (f). There is little error in the calculation results. Further, the graph of the correlation amount C (k) shown in FIG. 19 (c) also shows a sharp minimum value with a true image shift amount, and the image shift amount indicated by a cross in FIGS. 20 (e) and 20 (f). There is little error in the calculation results.

  As described above, the correlation amount C1 (k) of the calculation parameter spn = 1 can be expected to detect image displacement with high accuracy for high-frequency data, but the image displacement detection accuracy for low-frequency data. Decreases. On the other hand, the correlation amount C2 (k) of the calculation parameter spn = 10 can be expected to detect image displacement with high accuracy for low-frequency data, but the image displacement detection accuracy decreases for high-frequency data. When the data cycle matches the calculation parameter spn (= 10), the reliability of the image shift detection result is lost.

  On the other hand, as shown in equation (9), C (k), which is the correlation amount obtained by adding the partial correlation amounts of the calculation parameters spn = 1 and 10 by taking absolute values and integrating the added values, Image shift detection can be performed with accuracy of a predetermined level or higher for any low-frequency data. Therefore, if the correlation amount C (k) expressed by equation (9) is always used, stable image shift detection can be performed regardless of the spatial frequency component of the data, and the spatial frequency component of the data is detected and detected. The trouble of changing the value of the calculation parameter spn in accordance with the spatial frequency component can be saved.

Returning to FIG. 12 again, the description will be continued. In step S230, the graph of the correlation amount C (k) obtained by the correlation calculation equation shown in the equation (7) shows a shift amount (FIG. 21 (a) ), The correlation amount C (k) is the smallest (the smaller the correlation is, the higher the correlation is) at k = k _j = 2). Using a three-point interpolation method according to equations (10) to (13), a shift amount x that gives a minimum value C (x) for a continuous correlation amount is obtained.
x = kj + D / SLOP (10),
C (x) = C (kj) − | D | (11),
D = {C (k _j −1) −C (k _j +1)} / 2 (12),
SLOP = MAX {C ( _kj + 1) -C ( _kj ), C ( _kj- 1) -C ( _kj )}
... (13)

In step S240, the defocus amount DEF of the subject image plane with respect to the scheduled image formation plane can be obtained by the following equation using the shift amount x obtained by equation (10).
DEF = KX · PY · x (14)
In equation (14), PY is a detection pitch, and KX is a conversion coefficient determined by the size of the opening angle of the center of gravity of the pair of distance measuring pupils.

Whether or not the calculated defocus amount DEF is reliable is determined as follows. As shown in FIG. 21B, when the degree of correlation between a pair of data is low, the value of the minimum correlation value C (x) interpolated is increased. Therefore, it is determined that the reliability is low when C (x) is equal to or greater than a predetermined value. Alternatively, in order to normalize C (x) with the contrast of data, if the value obtained by dividing C (x) by SLOP that is proportional to the contrast is equal to or greater than a predetermined value, it is determined that the reliability is low. Alternatively, if SLOP that is proportional to the contrast is equal to or less than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated defocus amount DEF is low. As shown in FIG. 21C, when the correlation between the pair of data is low and there is no drop in the correlation amount C (k) between the shift ranges k _{min to} k _max , the minimum value C (x) is obtained. In such a case, it is determined that the focus cannot be detected. When focus detection is possible, the defocus amount is calculated by multiplying the calculated image shift amount by a predetermined conversion coefficient.

  In step S250, the focus detection calculation process (correlation calculation process) is terminated, and the process returns to step S140 in FIG.

<< Other Embodiments of the Invention >>
In the embodiment described above, as shown in the expression (7), the absolute values of the partial correlation amounts obtained by the different calculation parameters spn in the correlation calculation expression of the same format are taken and added, and the added value is obtained. Image shift detection is performed using the accumulated correlation amount C (k), but the calculation parameter spn may be a combination other than 1 and 10, or a combination of three or more calculation parameters spn. Further, the correlation calculation formula is not limited to the formula shown in the formula (7), and for example, a formula in which the calculation parameter spn of the correlation calculation formula as shown in the formula (17) is made different can be adopted.

Next, an embodiment in which image shift detection is performed using a plurality of correlation calculation expressions of different formats will be described. As a representative example of the correlation calculation expression, the calculation expression surrounded by the absolute value on the right side of the expression (15) is a correlation calculation expression A, and the calculation expression surrounded by the absolute value of the right side of the expression (16) is a correlation calculation expression B. Then, the operation expression surrounded by the absolute value on the right side of the expression (17) will be described as a correlation operation expression C, and their characteristics will be described.
C (k) = Σ | A _i −B _{i + k} | (15),
C (k) = Σ | Ai · B _{i + spn + k} −B _{i + k} · A _{i + spn} | (16),
C (k) = Σ | A _i ² · B _{i-spn + k} · B _{i + spn + k} −B _{i + k} ² · A _i-sp
n · A _{i + spn} | (17)
In the equations (15) to (17), the integration operation (Σ) is performed by sequentially moving the suffix i over a predetermined section of the data string.

  FIG. 22 is a table comparing the characteristics of the correlation calculation formulas A, B, and C described above. When there is no vignetting (when a pair of images are not distorted by vignetting), it is possible to detect image misalignment with high accuracy by any arithmetic expression, but the correlation arithmetic expression A is a multiplication between data during the calculation. In addition to the advantage that the calculation processing time can be shortened, noise resistance (characteristic that the image shift detection result is less affected even when noise is added to the data) is also better than the correlation calculation formulas B and C. high.

  When zero-order vignetting occurs, the correlation calculation formulas B and C can detect the image shift with high accuracy, but the calculation formula A decreases the image shift detection accuracy. In terms of noise resistance, the calculation formula B is higher than the calculation formula C. When primary vignetting has occurred, the correlation calculation formula C can detect image shift with high accuracy, but the calculation formulas A and B have low image shift detection accuracy.

  23 to 28 are simulation data representing the characteristics of the above-described correlation calculation formulas A, B, and C. FIG. FIG. 23 is a graph of a pair of image data (represented by ■ and □ in the figure), with the horizontal axis representing the data position and the vertical axis representing the data value. However, in FIG. 23, one of the pair of image data is represented by being shifted by one data position with respect to the other in order to prevent the pair of image data from being overlapped and difficult to understand.

  FIG. 23A is a graph in the case where there is no vignetting between a pair of image data (sin waveform) (when the images completely coincide with each other). FIG. 23B is a graph in the case where random noise is added to the pair of image data (sin waveform) in FIG. Further, FIG. 23C is a graph in the case where zero-order vignetting occurs between a pair of image data (sin waveform) and random noise is added. FIG. 23D is a graph when primary vignetting occurs between a pair of image data (sin waveform) and random noise is added.

  FIG. 24 shows a case where image shift detection is performed by applying the correlation arithmetic expressions A, B, and C in an image shift state where the pair of image data shown in FIG. 23 is relatively shifted from −1 to +1 of the data pitch. Is a graph of the calculation results (in the figure, the ■ indicates the result of the correlation calculation expression A, the □ indicates the result of the correlation calculation expression B, and the x indicates the result of the correlation calculation expression C). The image shift amount (unit: data pitch) is shown on the vertical axis, and the vertical axis represents the image shift amount of the calculation result. 24 (a), (b), (c), and (d) are calculation results of image shift amounts corresponding to the image data in FIGS. 23 (a), (b), (c), and (d), respectively. is there.

  FIG. 25 shows an error between the actual image shift amount shown in FIG. 24 and the image shift amount of the calculation result (in the figure, ■ indicates an error of the correlation calculation formula A, □ indicates an error of the correlation calculation formula B, × Is a graph of the correlation calculation formula C, and the horizontal axis represents the actual image shift amount (unit: data pitch), and the vertical axis represents the error amount (unit: data pitch). 25 (a), (b), (c), and (d) are error amounts corresponding to the calculation results of FIGS. 24 (a), (b), (c), and (d), respectively.

  FIG. 26 is a graph of the correlation amount C (k) when the correlation calculation formula A is applied in an image shift state where the pair of image data shown in FIG. 23 is relatively shifted by the data pitch −1. Represents the shift amount k (unit: data pitch), and the vertical axis represents the value of the correlation amount C (k). 26 (a), (b), (c), and (d) show the correlation amounts C (k) corresponding to the image data in FIGS. 23 (a), (b), (c), and (d), respectively. It is a graph.

  FIG. 27 is a graph of the correlation amount C (k) when the correlation calculation formula B is applied in an image shift state in which the pair of image data shown in FIG. 23 is relatively shifted by the data pitch −1. Represents the shift amount k (unit: data pitch), and the vertical axis represents the value of the correlation amount C (k). 27 (a), (b), (c), and (d) show the correlation amounts C (k) corresponding to the image data in FIGS. 23 (a), (b), (c), and (d), respectively. It is a graph.

  FIG. 28 is a graph of the correlation amount C (k) when the correlation calculation expression C is applied in an image shift state in which the pair of image data shown in FIG. 23 is relatively shifted by the data pitch −1. Represents the shift amount k (unit: data pitch), and the vertical axis represents the value of the correlation amount C (k). 28 (a), (b), (c), and (d) show the correlation amounts C (k) corresponding to the image data in FIGS. 23 (a), (b), (c), and (d), respectively. It is a graph.

  The characteristics of the correlation calculation expressions A, B, and C will be described again with reference to FIGS. For a pair of image data without vignetting shown in FIG. 23 (a), the correlation calculation formulas A, B, and C are all the correlations shown in FIGS. 26 (a), 27 (a), and 28 (a). The graph shows the true image shift amount and the sharp minimum value of the drop. As shown in FIGS. 24A and 25A, there is little error in the calculation result of the image shift amount.

  For image data in which noise is added to a pair of image data without vignetting shown in FIG. 23 (b), the correlation calculation formulas A, B, and C are all shown in FIGS. 26 (b), 27 (b), and FIG. As shown in FIG. 28 (b), the correlation graph shows a sharp image minimum with a true image shift amount. As shown in FIGS. 24 (b) and 25 (b), the images of the correlation arithmetic expressions A and B are shown. The error of the calculation result of the shift amount is small, but the error of the calculation result of the image shift amount of the correlation calculation formula C is slightly increased.

  For the image data in which noise is added to the pair of image data in which the 0th-order vignetting shown in FIG. 23 (c), the correlation equations B and C are shown in FIGS. 27 (c) and 28 (c). As shown in FIG. 24, the correlation graph shows a true image shift amount and a sharp minimum, and an error in the calculation result of the image shift amount of the correlation equation B as shown in FIGS. 24 (c) and 25 (c). Although there are few, the error of the calculation result of the image shift amount of the arithmetic expression C slightly increases.

  On the other hand, for the image data in which noise is added to the pair of image data in which the zero-order vignetting shown in FIG. 23 (c) occurs, the correlation graph is true in the correlation calculation expression A as shown in FIG. As shown in FIGS. 24 (c) and 25 (c), the calculation result of the image shift amount has a large error.

  For the image data in which noise is added to the pair of image data in which the primary vignetting shown in FIG. 23 (d), in the correlation calculation formula C, the correlation graph is a true image as shown in FIG. 28 (d). The amount of deviation shows a sharp minimum value, and as shown in FIGS. 24 (d) and 25 (d), the error in the calculation result of the image deviation amount of the correlation calculation formula C is relatively small.

  For the image data in which noise is added to the pair of image data in which the primary vignetting shown in FIG. 23 (d), the correlation graph in the correlation calculation formula B is true as shown in FIG. 27 (d). A sharp drop is shown in the vicinity of the shift amount, but the calculation result of the image shift amount has a large error (translation component) as shown in FIGS.

  On the other hand, for the image data in which noise is added to the pair of image data in which the primary vignetting shown in FIG. 23 (d), the correlation graph is true as shown in FIG. As shown in FIGS. 24 (d) and 25 (d), the calculation result of the image shift amount has a large error.

  Next, the compatibility between the correlation equation and the spatial frequency of the image data will be described. FIG. 29 is a table comparing the relationship between the calculation parameter spn and the frequency component of data in the correlation calculation formulas B and C. As can be seen from this table, when a pair of image data includes many low-frequency components, a larger value of the calculation parameter spn in the correlation calculation formulas B and C can be expected to provide a more accurate image shift detection result. On the other hand, when many high-frequency components are included, a smaller value of the calculation parameter spn in the correlation calculation formulas B and C can be expected to provide a highly accurate image shift detection result.

  30 and 31 are simulation data for representing the characteristics of the calculation parameter spn in the correlation calculation formulas B and C. FIG. FIG. 30 is a graph of a pair of image data (represented by ■ and □ in the figure), with the horizontal axis representing the data position and the vertical axis representing the data value. However, in FIG. 30, one of the pair of image data is represented by being shifted by one data position with respect to the other in order to prevent the pair of image data from being overlapped and difficult to understand. FIG. 30A is a graph when primary vignetting occurs between a pair of image data (high-frequency sin waveform), and FIG. 30B is a graph between a pair of image data (low-frequency sin waveform). It is a graph when primary vignetting occurs.

  31 (a) and 31 (b) show the correlation calculation formula described above in the image shift state in which the pair of image data shown in FIGS. 30 (a) and 30 (b) is relatively shifted from −1 to +1 of the data pitch. Graph of calculation results when image shift detection is performed by applying B (spn = 1, 10) (when the ■ mark in the figure is spn = 1, the □ mark is when spn = 10, respectively) The horizontal axis represents the actual image shift amount (unit: data pitch), and the vertical axis represents the calculated image shift amount.

  FIG. 31 (c) shows the above-described correlation operation formula C (spn = 1, in the image shift state where the pair of image data shown in FIG. 30 (b) is relatively shifted from −1 to +1 of the data pitch. 10) is a graph of calculation results when image shift detection is performed (in the figure, the ■ mark represents spn = 1, the □ mark represents spn = 10, respectively), and the horizontal axis represents The actual image shift amount (unit: data pitch), and the vertical axis represents the calculated image shift amount.

  Further, FIG. 31 (d) shows the correlation calculation formula C described above in an image shift state where the data pitch is relatively shifted from −1 to +1 after adding noise to the pair of image data shown in FIG. 30 (b). FIG. 6 is a graph of calculation results when image misalignment detection is performed by applying (spn = 1, 10) (in the figure, the case where ■ mark is spn = 1, and the case where □ mark is spn = 10) Yes, the horizontal axis represents the actual image shift amount (unit: data pitch), and the vertical axis represents the calculated image shift amount.

  As is clear from FIGS. 31A and 31B, according to the correlation calculation formula B, the calculation parameter spn is adjusted according to the frequency of the image data (spn is increased in the case of a low frequency, and is increased in the case of a high frequency. By reducing spn, the shift phenomenon of the image shift detection calculation result due to primary vignetting can be reduced.

  As is clear from FIGS. 31 (c) and 31 (d), according to the correlation calculation expression C, the calculation parameter spn is adjusted according to the frequency of the image data (in the case of low frequency, spn is increased). Therefore, it is possible to reduce the variation phenomenon of the image shift detection calculation result due to noise.

  In general, according to the correlation calculation formula including multiplication of data as in correlation calculation formulas B and C, the interval between data (= spn) is adjusted according to the frequency of the image data (in the case of low frequency, spn is adjusted. The effect of vignetting and noise can be reduced by reducing spn in the case of large and high frequency.

  Based on the above, the body drive control device 214 executes focus detection calculation processing according to the flow shown in FIG. Focus detection calculation processing is started from step S300, and in step S310, the data processing of the pair of focus detection pixels is subjected to filter processing for cutting high frequency components equal to or higher than the Nyquist frequency component according to equation (1), and the first data sequence A second data string is generated.

  In step S310, a plurality of correlation calculation formulas (A, B: small spn, B: large spn, C: small spn, C: large spn) are applied to the first data string and the second data string. The defocus amount is calculated according to the correlation calculation formula. In step S330, the reliability of the defocus amount obtained by each correlation calculation expression is determined according to the correlation parameters (C (x), SLOP) shown in FIG. 21, and a highly reliable defocus amount is extracted. .

  In step S340, the extracted defocus amounts are averaged, the correlation parameters corresponding to the extracted defocus amounts are averaged, and the focus detection calculation process is terminated.

Next, FIG. 33 is a flowchart showing a defocus amount calculation process for each correlation calculation formula (A, B: small spn, B: large spn, C: small spn, large C: spn). Each processing step is executed by the body drive control device 214. The defocus amount calculation processing is started from step S400, and in the subsequent step S410, the first data sequence (A _{1 to An} ) and the second data sequence (B _{1 to} B _n ) are relatively shifted (shift amount k). The correlation calculation formula (A, B: small spn, B: large spn, C: small spn, large C: spn) is applied to the first data string and the second data string, and the absolute value of the calculation result is data. Are accumulated over a predetermined interval.

  In step S420, the image shift amount is calculated based on the minimum value of the correlation amount C (k) by the three-point interpolation method described in FIG. 21, and the image shift amount is converted into a defocus amount in step S430. The defocus amount calculation process ends.

  In the embodiment described above, first, a plurality of defocus amounts corresponding to a plurality of correlation calculation expressions are calculated, then reliable defocus amounts are extracted, and the extracted defocus amounts are averaged. As a result, the final defocus amount is calculated, enabling stable and highly accurate focus detection regardless of the vignetting state, noise state, and spatial frequency components of the data, as well as the vignetting state and noise. Defocus amount by detecting the spatial frequency component of the data and the spatial frequency component of the data, and selecting the optimal correlation equation from multiple correlation equations according to the detected vignetting state, noise state, and spatial frequency component of the data Compared with the calculation of vignetting, it is possible to omit the process of detecting the vignetting state, the noise state, and the spatial frequency component of the data, and at the same time, the vignetting state, the noise state and the spatial frequency component of the data. It is possible to prevent the influence of erroneous detection.

  In the processing shown in FIG. 32 and FIG. 33, an example of calculating up to the defocus amount for each correlation calculation expression is shown. However, the image shift amount is calculated for each correlation calculation expression, and the image shift amount is calculated according to the reliability. After the extraction and obtaining the average of the extracted image deviation amounts, the average image deviation amount may be converted into a defocus amount.

  In the process shown in FIG. 32, an example is shown in which a plurality of defocus amounts corresponding to a plurality of correlation calculation expressions are calculated, and then a reliable defocus amount is extracted from them. As in the detection calculation process, image shift detection using a plurality of correlation calculation expressions may be performed in a predetermined order, and the image shift detection process may be terminated when a reliable defocus amount is obtained. .

  Each processing step in FIG. 34 is executed by the body drive control device 214. In step S500, focus detection calculation processing is started, and in step S510, the data sequence of the pair of focus detection pixels is subjected to filter processing for cutting high frequency components equal to or higher than the Nyquist frequency component according to the equation (1), and the first data sequence A second data string is generated. In step S520, the defocus amount is calculated by applying the correlation calculation expression A to the first data string and the second data string.

  In step S530, it is checked whether or not the calculated defocus amount is reliable. If the calculated defocus amount is reliable, the process proceeds to step S610 to end the focus detection calculation process. On the other hand, if there is no reliability, the process proceeds to step S540, and the defocus amount is calculated by applying the correlation operation formula B (small spn) to the first data string and the second data string.

  In step S550, it is checked whether or not the calculated defocus amount is reliable. If the defocus amount is reliable, the process proceeds to step S610 to end the focus detection calculation process. On the other hand, if there is no reliability, the process proceeds to step S560, and the defocus amount is calculated by applying the correlation calculation formula B (large spn) to the first data string and the second data string.

  In step S570, it is checked whether or not the calculated defocus amount is reliable. If the defocus amount is reliable, the process proceeds to step S610 to end the focus detection calculation process. On the other hand, if there is no reliability, the process proceeds to step S580, and the defocus amount is calculated by applying the correlation calculation formula C (small spn) to the first data string and the second data string.

  In step S590, it is checked whether or not the calculated defocus amount is reliable. If the defocus amount is reliable, the process proceeds to step S610 to end the focus detection calculation process. On the other hand, if there is no reliability, the process proceeds to step S600, and the defocus amount is calculated by applying the correlation calculation expression C (large spn) to the first data string and the second data string. In step S610, the focus detection calculation process is performed. Exit.

  According to the focus detection calculation process shown in FIG. 34, a plurality of correlation calculation formulas are in order of high image shift detection accuracy and small calculation scale with respect to general data having no vignetting or noise and including high-frequency components. Since it is applied to (correlation calculation formula A → Bspn small → Bspn large → Cspn small → Cspn large), it is possible to perform focus detection with high accuracy in a short time compared to the processing shown in FIG.

  According to the focus detection process shown in FIG. 34, an example is shown in which the defocus amount is calculated for each correlation calculation expression. However, the image shift amount is calculated for each correlation calculation expression, and the calculated image shift amount is trusted. The image shift amount calculated only when reliability is reliable may be converted into a defocus amount.

  In the embodiment described above, an example using a plurality of different correlation calculation expressions A, Bspn small, Bspn large, Cspn small, and Cspn large shown in the equations (15) to (17) is shown. Are not limited to the correlation calculation formulas shown in the formulas (15) to (17), and correlation calculation formulas expressed in other formats can also be used.

  In the focus detection calculation process shown in FIG. 12, the partial correlation amounts calculated with different calculation parameters spn in the correlation calculation formula of the same format are added as absolute values, and the added values are integrated over a predetermined data section. In this example, the correlation amount C (k) is calculated, but each of the partial correlation amounts calculated using different types of correlation calculation expressions (for example, the expressions (15), (16), and (17)) is absolute. The correlation amount C (k) may be obtained by taking the values and adding them and integrating the added values over a predetermined data interval. In that case, it is possible to standardize by dividing the partial correlation amount calculated by each correlation calculation expression by a predetermined value corresponding to each correlation calculation expression, and to align the level of the partial correlation amount. As the predetermined value corresponding to each correlation calculation expression, for example, the maximum value of the correlation amount C (k) obtained by applying each correlation calculation expression to a standard pair of data can be adopted.

<< Other modifications >>
The arrangement of the focus detection areas in the image sensor is not limited to that shown in FIG. 2, and the focus detection areas can be arranged in the diagonal direction and in other positions in the horizontal and vertical directions.

  In the imaging device 212 illustrated in FIG. 3, the focus detection pixels 313 and 314 are provided with one photoelectric conversion unit in one pixel, but a pair of photoelectric conversion units is provided in one pixel. Also good. FIG. 35 is a partially enlarged view of the image sensor 212A corresponding to the image sensor 212 shown in FIG. 3, and the focus detection pixel 311 includes a pair of photoelectric conversion units in one pixel. The focus detection pixel 311 shown in the drawing performs a function corresponding to the pair of the focus detection pixel 313 and the focus detection pixel 314 shown in FIG.

  The focus detection pixel 311 includes a microlens 10 and a pair of photoelectric conversion units 13 and 14 as shown in FIG. The focus detection pixel 311 is not provided with a color filter in order to increase the amount of light, and its spectral characteristic is a spectral that combines the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristic of an infrared cut filter (not shown). Characteristics (see FIG. 7). That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 6, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, the red pixel, and the blue pixel. ing.

  FIG. 37 is a diagram for explaining the focus detection operation of the pupil division type phase difference detection method by the focus detection pixels of the image sensor 212A shown in FIG. In FIG. 37, reference numeral 90 denotes an exit pupil set at a distance d ahead of the microlens arranged on the planned imaging plane of the interchangeable lens. Here, the distance d is a distance determined according to the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and the distance measurement pupil distance. 91 is an optical axis of the interchangeable lens, 50 and 60 are microlenses, (53, 54), (63, 64) are photoelectric conversion units of a pair of focus detection pixels, and 73, 74, 83, and 84 are focus detection light beams. is there.

  Reference numeral 93 denotes an area of the photoelectric conversion units 53 and 63 projected by the microlenses 50 and 60, which is a distance measuring pupil. Similarly, 94 is an area of the photoelectric conversion units 54 and 64 projected by the microlenses 50 and 60, and is a distance measuring pupil. In FIG. 37, a focus detection pixel (having a microlens 50 and a pair of photoelectric conversion units 53 and 54) on the optical axis 91 and an adjacent focus detection pixel (a microlens 60 and a pair of photoelectric conversion units 63 and 64) are arranged. In the focus detection pixels arranged in the periphery on the imaging surface, the pair of photoelectric conversion units arrive at each microlens from the pair of distance measuring pupils 93 and 94, respectively. Receives light flux. The arrangement direction of the focus detection pixels is made to coincide with the arrangement direction of the pair of distance measurement pupils.

  The microlenses 50 and 60 are disposed in the vicinity of the planned imaging plane of the optical system, and the shape of the pair of photoelectric conversion units 53 and 54 disposed behind the microlens 50 disposed on the optical axis 91 is formed. Projection is performed on the exit pupil 90 separated from the microlenses 50 and 60 by the distance measurement pupil distance d, and the projection shape forms distance measurement pupils 93 and 94. Further, the microlens 60 disposed adjacent to the microlens 50 projects the shape of the pair of photoelectric conversion units 63 and 64 disposed behind the microlens 50 onto the exit pupil 90 separated by the distance measuring pupil distance d. The projection shape forms distance measuring pupils 93 and 94. That is, the positions of the microlens and the photoelectric conversion unit of each pixel so that the projection shapes (ranging pupils 93 and 94) of the photoelectric conversion unit of each focus detection pixel coincide on the exit pupil 90 at the distance measurement pupil distance d. The relationship has been determined.

  The photoelectric conversion unit 53 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 73 passing through the distance measuring pupil 93 and traveling toward the microlens 50. In addition, the photoelectric conversion unit 54 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 74 that passes through the distance measuring pupil 94 and travels toward the microlens 50. Similarly, the photoelectric conversion unit 63 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 83 that passes through the distance measuring pupil 93 and travels toward the microlens 60. In addition, the photoelectric conversion unit 64 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 84 that passes through the distance measuring pupil 94 and travels toward the microlens 60.

  A large number of such focus detection pixels are arranged in a straight line, and the output of the pair of photoelectric conversion units of each focus detection pixel is grouped into an output group corresponding to the distance measurement pupil 93 and the distance measurement pupil 94, thereby the distance measurement pupil 93. And information on the intensity distribution of the pair of images formed on the focus detection pixel array by the focus detection light fluxes passing through the distance measuring pupil 94. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, the image shift amount of a pair of images is detected by a so-called pupil division phase difference detection method. Further, by applying a predetermined conversion process to the image shift amount, the current image plane (actual image plane at the focus detection position determined on the photographing screen 100) with respect to the planned image plane (the position of the microlens array). Deviation (defocus amount) is calculated.

  Next, in the image sensor 212 shown in FIG. 3, the example in which the imaging pixel 310 includes a Bayer color filter is shown. However, the configuration and arrangement of the color filter are not limited to this, and the complementary color filter (green: G, yellow: Ye, magenta: Mg, cyan: Cy) may be employed. Further, in the image pickup device 212 shown in FIG. 3, an example in which the focus detection pixels 313 and 314 are not provided with a color filter is shown. However, one of the color filters of the same color as the image pickup pixel 310 (for example, a green filter) is provided. Even in this case, the present invention can be applied.

  In the focus detection pixels 311, 313, and 314 shown in FIGS. 5 and 37 of the embodiment described above, an example in which the shape of the photoelectric conversion unit is semicircular or rectangular is shown. The shape of the part is not limited to these, and may be other shapes. For example, the shape of the photoelectric conversion unit of the focus detection pixel can be an ellipse or a polygon.

  Furthermore, in the imaging device 212 shown in FIG. 3, the example in which the imaging pixels and the focus detection pixels are arranged in a dense square lattice arrangement is shown, but a dense hexagonal lattice arrangement (honeycomb arrangement) may be used.

  In the above-described embodiment, the focus detection operation by the pupil division type phase difference detection method using the microlens has been described. However, the present invention is not limited to the focus detection of such a method, and Japanese Patent Application Laid-Open No. 2008-15157. The present invention is also applicable to a focus detection apparatus of a pupil division type phase difference detection method using a polarizing element disclosed in the publication.

  Furthermore, the present invention can also be applied to focus detection by a pupil division type phase difference detection method using a re-imaging method. The focus detection operation of the pupil division type phase difference detection method by the re-imaging method will be described with reference to FIG. 38, 191 is the optical axis of the interchangeable lens, 110 and 120 are condenser lenses, 111 and 121 are aperture masks, 112, 113, 122 and 123 are aperture openings, 114, 115, 124 and 125 are re-imaging lenses, Reference numerals 116 and 126 denote image sensors (CCD) for focus detection.

  Reference numerals 132, 133, 142, and 143 denote focus detection light beams. Reference numeral 190 denotes an exit pupil set at a distance d5 in front of the planned imaging plane of the interchangeable lens. Here, the distance d5 is a distance determined according to the focal length of the condenser lenses 110 and 120, the distance between the condenser lenses 110 and 120 and the aperture openings 112, 113, 122, 123, and the like. Distance. Reference numeral 192 denotes a region of the aperture openings 112 and 122 projected by the condenser lenses 110 and 120, which is a distance measuring pupil. Similarly, reference numeral 193 denotes a region of the aperture openings 113 and 123 projected by the condenser lenses 110 and 120, which is a distance measuring pupil. Condenser lens 110, diaphragm mask 111, diaphragm apertures 112 and 113, re-imaging lenses 114 and 115, and image sensor 116 perform focus detection in a re-imaging type pupil division type phase difference detection method in which focus detection is performed at one position. Configure the unit.

  FIG. 38 schematically illustrates the focus detection unit on the optical axis 191 and the focus detection unit outside the optical axis. By combining a plurality of focus detection units, it is possible to realize a focus detection apparatus that performs focus detection by pupil division phase difference detection of the re-imaging method at the three focus detection positions 101 to 103 shown in FIG.

  The focus detection unit having the condenser lens 110 is disposed between the condenser lens 110 disposed in the vicinity of the planned imaging surface of the interchangeable lens, the image sensor 116 disposed behind the condenser lens 110, and between the condenser lens 110 and the image sensor 116. A pair of re-imaging lenses 114 and 115 for re-imaging the primary image formed in the vicinity of the scheduled imaging surface on the image sensor 116, and the vicinity of the pair of re-imaging lenses (front surface in FIG. 38). A diaphragm mask 11 having a pair of diaphragm openings 112 and 113 is provided.

  The image sensor 116 is a line sensor in which a plurality of photoelectric conversion units are densely arranged along a straight line, and the arrangement direction of the photoelectric conversion units is made to coincide with the dividing direction of the pair of distance measuring pupils (= aperture aperture arrangement direction). . Information corresponding to the intensity distribution of the pair of images re-imaged on the image sensor 116 is output from the image sensor 116, and image shift detection calculation processing (correlation processing, phase difference detection described later) is performed on this information. By performing the processing, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method (re-imaging method). Further, the deviation (defocus amount) of the current imaging plane with respect to the planned imaging plane is calculated by multiplying the image shift amount by a predetermined conversion coefficient.

  The image sensor 116 is projected onto the planned imaging plane by the re-imaging lenses 114 and 115, and the detection accuracy of the defocus amount (image deviation amount) is determined by the detection pitch of the image deviation amount (in the case of the re-imaging method). This is determined by the arrangement pitch of the photoelectric conversion units projected on the planned imaging plane.

  The condenser lens 110 projects the aperture openings 112 and 113 of the aperture mask 111 as areas 192 and 193 on the exit pupil 190. Regions 192 and 193 are distance measuring pupils. That is, a pair of images re-imaged on the image sensor 116 is formed by a light beam passing through the pair of distance measuring pupils 192 and 193 on the exit pupil 190. The light beams 132 and 133 that pass through the pair of distance measuring pupils 192 and 193 on the exit pupil 190 are referred to as focus detection light beams.

  Even in such a pupil division type phase difference detection method based on the re-imaging method, the balance of the pair of images formed on the image sensor is lost due to vignetting of the distance measuring pupil, so the output signal of the image sensor is processed. In this case, the present invention can be applied.

  Note that the imaging apparatus is not limited to a digital still camera or a film still camera in which an interchangeable lens is mounted on the camera body as described above. For example, the present invention can be applied to a lens-integrated digital still camera, film still camera, or video camera. Furthermore, the present invention can be applied to a small camera module built in a mobile phone, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, and the like.

  In the above-described embodiment, the focus detection pixels are arranged in the array of the imaging pixels, and a pair of light beams corresponding to a pair of light beams passing through different regions of the pupil of the imaging optical system based on a signal obtained by the focus detection pixels. Although an example of obtaining a signal sequence and performing focus detection by the pupil division type phase difference detection method by the above correlation calculation has been shown, well-known contrast method focus detection is performed based on the output of the imaging pixel, and output of the focus detection pixel It is also possible to adopt a hybrid focus detection device that performs focus detection using a pupil division type phase difference detection method based on the above.

  The present invention can also be applied to focus detection devices other than cameras, distance measuring devices, and stereo distance measuring devices. Furthermore, the present invention can also be applied to an apparatus that detects the movement of a subject image and camera shake by detecting the correlation between signals from image sensors having different times. Furthermore, the present invention can be applied to pattern matching between an image signal of an image sensor and a specific image signal.

  Furthermore, the present invention is not limited to the one that detects the correlation between the image signal data, but can also be applied to the one that detects the correlation between the data related to sound and generally the correlation between two signals.

DESCRIPTION OF SYMBOLS 10; Micro lens, 11, 13, 14; Photoelectric conversion part, 202; Interchangeable lens, 212, 212A; Image pick-up element, 214; Body drive control apparatus, 310; Image pick-up pixel, 311, 313, 314;

Claims (12)

An imaging device having a plurality of pixels provided with a first photoelectric conversion unit that receives one of the pair of light beams that have passed through different pupil regions of the imaging optical system and a second photoelectric conversion unit that receives the other light beam; ,
The first data sequence from the plurality of first photoelectric conversion units and the second data sequence from the plurality of second photoelectric conversion units are relatively displaced while changing a displacement amount, and a plurality of correlation calculation expressions are used. A correlation calculation unit that calculates a plurality of correlation amounts between the first data string and the second data string and calculates an addition correlation amount obtained by adding the plurality of correlation amounts;
A lens drive control unit that converts the added correlation amount into a defocus amount and generates a control signal for driving the imaging optical system based on the defocus amount;
An imaging device comprising: The imaging device according to claim 1,
At least one of the plurality of correlation calculation formulas is a correlation calculation formula including multiplication of one data in the first data string and one data in the second data string. An imaging device that is characterized. The imaging device according to claim 2,
There are a plurality of correlation calculation expressions including the multiplication,
The correlation calculation expression including the plurality of multiplications is the multiplication of one data in the first data string and one data in the second data string, and the other data in the first data string. Including a partial correlation calculation expression including multiplication of one data and another data in the second data string,
An interval between one data in the first data string and another data in the first data string, and one data in the second data string and the second data string An image pickup apparatus characterized in that the interval between the other one data is a predetermined interval. The imaging device according to claim 3.
The correlation calculation formula including the plurality of multiplications includes two types of the partial correlation calculation formulas having different predetermined intervals. An imaging device having a plurality of pixels provided with a first photoelectric conversion unit that receives one of the pair of light beams that have passed through different pupil regions of the imaging optical system and a second photoelectric conversion unit that receives the other light beam; ,
The first data sequence from the plurality of first photoelectric conversion units and the second data sequence from the plurality of second photoelectric conversion units are relatively displaced while changing a displacement amount, and a plurality of correlation calculation expressions are used. A correlation calculation unit for calculating a plurality of correlation amounts between the first data string and the second data string and obtaining a plurality of correlation amounts;
According to the correlation parameter obtained by the plurality of correlation calculation formulas, a highly reliable defocus amount is determined,
A body drive control unit that generates a control signal for driving the imaging optical system based on an averaged defocus amount of the defocus amount determined to be high in reliability ;
An imaging device comprising: The imaging apparatus according to claim 5,
The imaging apparatus , wherein the correlation parameter is at least one of a minimum correlation value and a value proportional to contrast . In the imaging device according to claim 5 or 6,
The plurality of correlation calculation formulas are calculation formulas having different characteristics with respect to distortion between the first data sequence and the second data sequence, respectively. The imaging apparatus according to claim 7,
A correlation operation including multiplication of one data in the first data string and one data in the second data string as a defocus amount determined to have high reliability when the distortion is large An imaging apparatus that extracts a correlation amount between the first data string and the second data string calculated by an expression. In the imaging device according to claim 5 or 6,
The plurality of correlation calculation formulas are calculation formulas having different characteristics with respect to noise applied to the first data sequence and the second data sequence, respectively. The imaging device according to claim 9,
Correlation calculation including multiplication of one data in the first data string and one data in the second data string as a defocus amount determined to have high reliability when the noise is large An imaging apparatus that extracts a correlation amount between the first data string and the second data string calculated by an expression. In the imaging device according to claim 5 or 6,
The imaging apparatus, wherein the plurality of correlation calculation formulas are calculation formulas having different characteristics with respect to spatial frequencies of the first data sequence and the second data sequence. 12. The imaging apparatus according to claim 1, further comprising a calculation unit configured to calculate the displacement amount from which an extreme value of the correlation amount is obtained.

Images