JP6677336B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging device.

  Accumulation control that splits a light beam from the imaging optical system in two directions, guides the light flux to an image sensor in which focus detection pixels are arranged in an array of imaging pixels, and independently controls a charge accumulation operation in each image sensor. An imaging device including the means has also been proposed (see Patent Document 1).

Japanese Patent No. 4858179

  An object of the present invention is to provide an imaging device capable of performing suitable charge accumulation control.

  The present invention solves the above problem by the following means.

  According to the present invention, a first image sensor that captures an image of a subject and outputs a first signal, a second image sensor that captures an image of the subject and outputs a second signal, and a signal output from the first image sensor A display unit that displays an image generated based on the first signal; a storage unit that stores an image generated based on the first signal output from the first image sensor; and a storage unit that stores the image. An imaging condition setting unit that sets an imaging condition of the first imaging device that generates an image to be performed based on the second signal output from the second imaging device, and a base of an image that is displayed on the display unit. A first imaging condition of the first imaging device that generates the first signal, and a second imaging condition of the second imaging device that generates the second signal used by the imaging condition setting unit, The change in the first imaging condition is smaller than the change in the second imaging condition. A control unit for controlling so as to reduce, to an image pickup apparatus having a.

  According to the present invention, it is possible to provide an imaging device capable of performing suitable charge accumulation control.

1 is a schematic diagram of a camera to which an embodiment of the present invention has been applied. FIG. 3 is an enlarged view of an imaging pixel array in the first imaging device. FIG. 9 is a diagram illustrating an arrangement of AF areas in a second image sensor. FIG. 5 is an enlarged view of a focus detection pixel column in a second image sensor. FIG. 2 is a block diagram of a camera control device. FIG. 4 is an explanatory diagram of accumulation control by an imaging control unit. 5 is a flowchart of control performed by the camera control device during shooting standby and shooting. 6 is a time chart of control performed by the camera control device during shooting standby and shooting.

Hereinafter, embodiments of the present invention will be described with reference to the drawings and the like.
FIG. 1 is a schematic diagram of a camera 1 to which an embodiment of the present invention is applied.
The camera 1 according to the present embodiment is a digital camera that records a subject image formed by a photographing lens 20 having an imaging optical system as electrical image information.
The camera 1 includes a photographing lens 20 and a camera body 10, and the photographing lens 20 is mounted on the camera body 10 via a mount 30.

The taking lens 20 includes a zooming lens 21, a focusing lens 23, a diaphragm 24, a lens drive control device 25, and the like.
The lens drive control device 25 includes peripheral components such as a microcomputer and a memory. The drive control of the focusing lens 23 and the aperture 24, the detection of the states of the zooming lens 21, the focusing lens 23 and the aperture 24, and the connection with a camera control device 50 described later. Transmission of lens information and reception of camera information are performed by communication.

The camera body 10 includes a pellicle mirror 11, a first image sensor 41, a second image sensor 42, an EVF (Electric View Finder) 12, a memory card 13, a camera controller 50, and the like.
The pellicle mirror 11 has a reflective film surface 11P, and the reflective film surface 11P is disposed at an angle of 45 ° with respect to the optical axis of a light beam incident from the photographing lens 20. The pellicle mirror 11 splits the light beam coming from the photographing lens 20 into a reflected light beam and a transmitted light beam at a ratio of 1: 1 and bends the reflected light beam at an angle of 90 ° upward with respect to the incident light beam in the figure.

The first imaging element 41 and the second imaging element 42 are, for example, photoelectric conversion elements such as a CCD and a CMOS, and convert a subject image formed on a light receiving surface thereof into an electric signal.
The first imaging element 41 is arranged on a predetermined imaging plane (imaging plane) of the imaging lens 20 that receives a light beam transmitted through the pellicle mirror 11. The first imaging element 41 exclusively captures a display image and a recorded image in the EVF 12 described later.

The second imaging element 42 is arranged on the other planned imaging plane (imaging plane) of the photographing lens 20 above the pellicle mirror 11 so as to receive the light beam reflected by the pellicle mirror 11. The second imaging element 42 performs imaging for obtaining control information such as auto focus (AF) and automatic exposure (AE).
The first image sensor 41 and the second image sensor 42 will be described later in detail.

  The EVF 12 includes a liquid crystal display device driving circuit 12A, a liquid crystal display element 12B, an eyepiece 12C, and the like. The EVF 12 displays the image picked up by the first image pickup device 41 on the liquid crystal display element 12B, and makes it visible to the photographer via the eyepiece 12C.

The camera controller 50 includes a microcomputer and peripheral components such as a memory, and reads and corrects image signals from the first image sensor 41 and the second image sensor 42, and communicates with the lens drive controller 25 (lens). It receives information, transmits camera information such as the defocus amount, detects the focus adjustment state (defocus amount) of the photographing lens 20, and controls the operation of the entire camera.
The camera control device 50 and the lens drive control device 25 communicate with each other via an electric contact 31 provided on the mount unit 30 to exchange various information such as lens information, defocus amount, and aperture information. Details of the camera control device 50 will be described later.

  When the shutter button (not shown) is pressed (release operation), the camera 1 having the above configuration is controlled by the camera control device 50 to obtain control information from imaging by the second image sensor 42, and based on the control information. Under the control, the subject image light is converted into an electric signal by the first image sensor 41, and the image data is recorded (photographed) on the memory card 13.

Next, the first image sensor 41, the second image sensor 42, and the camera control device 50 will be described in detail with reference to FIGS.
FIG. 2 is an enlarged view of an image pickup pixel array in the first image pickup device 41. FIG. 3 is a diagram showing an arrangement of the AF area 42A in the second image sensor 42. FIG. 4 is an enlarged view of the focus detection pixel array 43 in the second image sensor 42. FIG. 5 is a block diagram of the camera control device 50. FIG. 6 is an explanatory diagram of the accumulation control by the imaging control unit 501.

As shown in the enlarged view of FIG. 2, the first imaging element 41 has imaging pixels 41P arranged two-dimensionally. The imaging pixel 41P includes three types of pixels, each having a color filter, a green pixel 41PG, a red pixel 41PR, and a blue pixel 41PB, and these pixels are two-dimensionally arranged in a Bayer arrangement.
The first image sensor 41 photoelectrically converts a subject image formed by the image forming optical system of the photographing lens 20, transmitted through the pellicle mirror 11, and formed on the imaging surface, and outputs the image to the camera control device 50.

In the second image pickup device 42, the image pickup pixels 42P are arranged two-dimensionally, and a focus detection pixel column 43 is incorporated in a portion corresponding to the AF area 42A shown in a front view in FIG. FIG. 4 shows the AF area 42A in an enlarged manner.
The imaging pixel 42P is composed of three types of pixels, each having a color filter, a green pixel 42PG, a red pixel 42PR, and a blue pixel 42PB, and these pixels except for the AF area 42A (the focus detection pixel column 43). , And two-dimensionally arranged in a Bayer array.

As shown in FIG. 3, the AF area 42A is set at a total of 25 places, that is, 5 places in the vertical direction and 5 places in the horizontal direction within the imaging range of the second imaging element 42.
As shown in FIG. 4, the focus detection pixel array 43 forming the AF area 42A includes a focus detection pixel 43L having an opening on the left side and a focus detection pixel 43R having an opening on the right side arranged alternately. The microlenses arranged on the pixels 43L and 43R receive light passing through the right and left sides of the pupil of the photographing lens 20.
These focus detection pixels 43L and 43R do not include a color filter. Based on the detection signal of the focus detection pixel array 43, a defocus calculation unit 521 of the second image processing unit 520 described later performs focus detection by a pupil division phase difference method. This will be described in detail later.

  The second imaging element 42 photoelectrically converts the subject image formed by the optical system of the imaging lens 20, reflected by the pellicle mirror 11 and formed on the imaging surface, and outputs the image to the camera control device 50. The imaging information (detection signal) is used for defocus detection for focus control, color tracking, face detection, and light amount detection (photometry) for exposure control.

  Here, as described above, the first image sensor 41 captures the display image and the recorded image on the EVF 12 and obtains the control information from the image capture of the second image sensor 42. And the effective pixel range of the second image sensor 42 match. Since the subject observation image of the EVF 12 is generated from the signal of the first image sensor 41, it also matches the effective pixel range of the second image sensor 42.

As shown in FIG. 5, the camera control device 50 includes a camera control unit 500, a first image processing unit 510, and a second image processing unit 520.
The camera control unit 500 includes an imaging control unit 501, an area selection unit 502, an EVF display unit 503, a lens control unit 504, and the like.

The imaging control unit 501 performs charge accumulation start, termination, thinning, reading, gain control, and the like for the first imaging element 41 and the second imaging element 42.
Here, accumulation for EVF display by the first image sensor 41 and detection of control information by the second image sensor 42 (defocus detection for focus control, color tracking and face detection, and light amount for exposure control) Accumulation for detection) differs depending on the information to be acquired, such as the presence / absence of a color filter of a pixel, the saturation level of a color, and the like. For this reason, the imaging control unit 501 performs accumulation control for controlling the charge accumulation time and the gain to levels suitable for the respective functions according to the acquired control information.

  That is, EVF display accumulation for acquiring imaging information from the imaging pixel 41P of the first imaging element 41, and the focus detection pixel array 43 without the color filter of the second imaging element 42 (the focus detection pixels 43L, 43R) for defocus detection to acquire detection information, and for color tracking and face detection to acquire color information for subject tracking and face detection from the imaging pixel 42P with a color filter of the second image sensor 42. Optimum values of the charge accumulation time and the gain are different between the accumulation and the photometric accumulation for acquiring light amount information for photometry from the imaging pixel with color filter 42P of the second image sensor 42.

Therefore, as shown in an example in FIG. 6, in the EVF display accumulation for displaying the subject image on the EVF 12, the change width of the accumulation control is set to be small and the change frequency is set low from the viewpoint of visibility.
On the other hand, in accumulation for defocus detection, accumulation for color tracking / face detection, and accumulation for photometry, the change width of accumulation control is increased and the frequency of change is increased according to the respective functions to obtain the optimum level, thereby achieving highly accurate detection. Is possible.

The area selection unit 502 selects an area in a screen used for AF or AE based on calculation results of a color tracking calculation unit and a face detection calculation unit, which will be described later, and a user's designation.
The EVF display unit 503 drives the liquid crystal display device driving circuit 12A in the EVF 12 to display the image generated by the EV image generation unit 512 in the first image processing unit 510 described later on the liquid crystal display element 12B.

  The lens control unit 504 drives the focusing lens 23 in the photographing lens 20 based on the calculation result of the defocus calculation unit 521 in the second image processing unit 520 described later, and controls the aperture value based on the calculation result of the photometry calculation unit. Is transmitted to the lens drive control device 25 in the photographing lens 20, and the photographing lens 20 is controlled via the lens drive control device 25.

The first image processing unit 510 includes a captured image generation unit 511 and an EV (electronic view) image generation unit 512.
The photographed image generating unit 511 generates a photographed image from a signal of the first image sensor 41 at the time of photographing.
The EV image generation unit 512 generates an image to be displayed on the EVF 12 from a signal of the first image sensor 41 during subject observation.

The second image processing unit 520 includes a defocus calculation unit 521, a photometry calculation unit 522, a color tracking calculation unit 523, a face detection calculation unit 524, and an AF pixel interpolation unit 525.
The defocus calculation unit 521 calculates the defocus amount of the photographing lens 20 from the signal of the focus detection pixel array 43 in the second image sensor 42 by a correlation calculation.

That is, the defocus calculation unit 521 combines the outputs of the pair of photoelectric conversion units of the focus detection pixels 43L and 43R in the focus detection pixel column 43 into output groups corresponding to each of the left and right divided ranging pupils. Thereby, information on the intensity distribution of a pair of images formed on the focus detection pixel row by the focus detection light flux passing through each ranging pupil is obtained.
By performing image shift detection calculation processing (correlation calculation processing and phase difference detection processing) on these pieces of information, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method.
Further, a deviation (defocus amount) of the current imaging plane from the planned imaging plane is calculated by performing a conversion operation on the image shift amount in accordance with the distance between the centers of gravity of the pair of ranging pupils.

The photometry calculation unit 522 calculates the optimal exposure for subject shooting from the signal of the imaging pixel 42P of the second imaging element 42 in the area selected by the area selection unit 502, and determines the shutter speed and aperture. If the AF area where the main subject exists can be specified by the user's designation, color tracking, or face detection, the photometric calculation is performed with emphasis on that area.
The color tracking calculation unit 523 stores the color of the main subject first specified by the user from the signal of the imaging pixel 42P of the second image sensor 42, and performs template matching based on the color to determine the position of the main subject in the screen. To track.

The face detection calculation unit 524 detects the position in the screen where the face of the person exists from the signal of the imaging pixel 42P of the second imaging element 42.
The AF pixel interpolating unit 525 has a color filter at a position of an AF pixel (focus detection pixels 43L and 43R in the focus detection pixel array 43) having no color filter and different sensitivities, and having a peripheral color filter. It is obtained by interpolation from the signal of the imaging pixel 42P. This signal is used by the photometry calculation unit 522, the color tracking calculation unit 523, and the face detection calculation unit 524.

Next, with reference to FIGS. 7 and 8, control of the image pickup devices 41 and 42 at the time of shooting standby and shooting by the camera control device 50 will be described.
FIG. 7 is a control flowchart of the camera control device 50 at the time of shooting standby and at the time of shooting. FIG. 8 is a time chart of control performed by the camera control device 50 at the time of shooting standby and shooting.

First, the entire flow of the control at the time of shooting standby and at the time of shooting will be described with reference to the flowchart shown in FIG. In the drawings and in the following description, “step” is also abbreviated as “S”.
When the power of the camera 1 is turned on, the camera control device 50 controls accumulation of the first image sensor (accumulation for EVF display), starts display by the EVF 12 (S701), and depresses a shutter button (not shown) (halfway). Press) (S702).

If it is determined in step 702 that the shutter button has been half-pressed, control information is obtained and their calculations are performed (S703).
The acquisition and computation of the control information in step 703 are accumulation and computation for defocus detection, accumulation and computation for color tracking and face detection, and accumulation and computation for photometric detection. As described above, the acquisition of these control information is performed by the imaging control unit 501 of the camera control unit 500 by controlling the accumulation of the second image element, and the calculation of the control information is performed by the second image processing unit 520.

  Next, focusing control is performed based on the defocus calculation result in step 703. That is, the lens control unit 504 transmits the defocus information to the lens drive control device 25, and drives the focusing lens 23 by the lens drive control device 25 to focus on the AF area 42A where the user designated or the main subject is present ( S704).

Then, it waits for a shooting command by fully pressing the shutter button (S705).
If it is not determined in step 705 that the shutter button is fully pressed (No), the acquisition and calculation of the control information in step 703 are repeated. Here, the results of the color tracking calculation and the face detection calculation are reflected in the selection of the AF area 42A in the next focusing control.

  In step 705, when it is determined that a photographing command has been issued by fully pressing the shutter button (Yes), photographing is controlled. That is, the size of the aperture opening is controlled by driving the aperture 24 via the lens drive control device 25 based on the photometric calculation result, and the accumulation control (accumulation for recording) of the first image sensor 41 is performed, and the imaging is performed. The information is recorded on the memory card 13 (S706).

Next, the control by the camera control device 50 will be described in more detail along a time series with reference to a time chart shown in FIG. FIG. 8 shows a part of the operation that is repeated several times from when the user half-presses the shutter button until the shutter button is fully pressed.
(Control related to first image sensor 41 before photographing)
T11: The power of the camera 1 is turned on. The first image sensor 41 starts accumulation for EVF display. At the time of the EVF display accumulation, accumulation control for variably controlling the accumulation time and the gain is performed according to the imaging pixel sensitivity of the first imaging element 41 and the subject luminance. The accumulation control is performed so that the brightness of the subject observation image does not frequently change significantly.
T12: End of EVF display accumulation. Start reading. The reading of the accumulation for EVF display is performed by thinning-out reading corresponding to the EVF display resolution.

T13: End of reading of the accumulation signal for EVF display. Start accumulation of EVF display for the next frame. The first image processing unit 510 starts an image generation process for displaying an EVF image.
T14: End the image processing for displaying the EVF image. Displayed on EVF12.
The accumulation, readout, image processing, and display for EVF display are repeated at predetermined time intervals.

(Control related to the second imaging element 42 before photographing)
T21: When the shutter button is half-pressed, accumulation of defocus detection for obtaining a signal for defocus detection from the second image sensor 42 is started. At the time of accumulation for lens control, accumulation control for variably controlling the accumulation time and gain in accordance with the sensitivity of the focus detection pixels 43L and 43R and the luminance of the subject is performed.
T22: End of accumulation for defocus detection. Start reading. In reading out the defocus detection accumulation signal, thinning-out reading is performed only on the row where the focus detection pixels 43L and 43R exist.

T23: End of reading of the accumulation signal for defocus detection. Start of color tracking and face detection accumulation for obtaining signals for color tracking and face detection from the second image sensor 42.
At the time of accumulation for color tracking / face detection, accumulation control is performed to variably control the accumulation time and the gain according to the sensitivity of the imaging pixel 42P and the luminance of the main subject so that the main subject portion is not saturated or blackened as much as possible.
Also, here, the focusing control based on the read out defocus detection accumulation signal is started.
The focusing control based on the accumulation signal for defocus detection performs defocus calculation by the defocus calculation unit 521 of the second image processing unit 520, and controls and drives the focusing lens 23 via the lens drive control device 25 to focus. Match.

T24: End of accumulation for color tracking / face detection. Start reading color accumulation / face detection accumulation.
T25: End of reading of the accumulation signal for color tracking / face detection. The accumulation for photometric calculation for obtaining a signal for photometric calculation from the second image sensor 42 is started. Further, the color tracking / face detection calculation based on the read color tracking / face detection information is started. Based on this result, the camera control unit selects an AF area for performing the next AF.
At the time of accumulation for photometric calculation, accumulation control is performed in accordance with the sensitivity of the imaging pixel 42P and the brightness of the subject so as to variably control the accumulation time and gain so that the entire screen is not saturated as much as possible.
T26: End of photometric calculation accumulation. Start reading.
T27: End of reading of the accumulation signal for photometry calculation. Start photometric calculation based on the read photometric calculation information. This result is used for controlling the shutter speed and the aperture when photographing is subsequently performed. If the photographing does not start, it is used for the next control of the accumulation time and gain of the second image sensor and the display of the shutter speed and aperture to the user.
After that, accumulation for lens control is started again, and the above accumulation is repeated until the shutter button is fully pressed.

(Control related to shooting operation)
T31: A shooting command by fully pressing the shutter button. The accumulation for photographing of the first image sensor 41 is started.
The accumulation time in the accumulation for photographing is controlled based on the exposure value obtained from the results of the previous photometry, color tracking, and face detection. Before the start of accumulation, the aperture 24 is reduced to a set value via the lens drive control device 25 of the photographing lens 20.
T32: End of accumulation for photographing. Start reading.
T33: End of reading of the accumulation signal for photographing. Image processing by the first image processing unit 510 and generation of a captured image are started.
T34: End of photographed image generation. The captured image is recorded on the memory card 13.

As described above, the present embodiment has the following effects.
(1) The camera 1 includes a first imaging element 41 having only imaging pixels 41P, a second imaging element 42 having imaging pixels 42P and focus detection pixels 43L and 43R, and performs first imaging. The display image and the recorded image on the EVF 12 are captured by the element 41, and control information such as defocus detection, color tracking / face detection, and photometry detection is acquired from the imaging of the second imaging element 42.
For this reason, accumulation control for electronic view images that cannot be changed frequently, and accumulation control for defocus detection, color tracking / face detection, and photometry detection can be performed independently. The signal can be acquired at the optimal accumulation level for For this reason, there is no performance degradation due to pixel output saturation, and the followability and accuracy can be improved.
(2) Since the recorded image is captured by the first image sensor having no focus detection pixels, there is no image quality degradation due to the AF pixel interpolation processing.

(Modified form)
The present invention is not limited to the embodiments described above, and various modifications and changes as described below are possible, and these are also within the scope of the present invention.
(1) In the above-described embodiment, the present invention is applied to a camera 1 in which a camera body and a lens barrel are formed separately and can be connected to each other and are interchangeable. However, the present invention is not limited to this, and may be applied to an integrated camera.

(2) In the above embodiment, the incident light beam is split in two directions by the pellicle mirror 11, but this light beam splitting optical element may use another configuration such as a light splitting prism. Further, the division angle and division ratio of the light beam are not limited to those of the present embodiment, but can be set as appropriate.
The embodiments and the modified embodiments can be used in appropriate combinations, but detailed description is omitted. Further, the present invention is not limited by the embodiments described above.

  1: camera, 10: camera body, 11: pellicle mirror, 12: EVF, 41: first imaging element, 41P: imaging pixel, 42: second imaging element, 42P: imaging pixel, 43: focus detection Pixel columns, 43L and 43R: focus detection pixels, 50: camera control device, 500: camera control unit, 501: imaging control unit, 510: first image processing unit, 511: captured image generation unit, 512: EV image Generation unit, 520: second image processing unit, 521: defocus calculation unit, 522: photometry calculation unit, 523: color tracking calculation unit, 524: face detection calculation unit, 525: AF pixel interpolation unit

Claims (12)

A first image sensor that captures an image of a subject and outputs a first signal;
A second image sensor that captures an image of the subject and outputs a second signal;
A display unit that displays an image generated based on the first signal output from the first image sensor;
A storage unit that stores an image generated based on the first signal output from the first image sensor,
An imaging condition setting unit configured to set an imaging condition of the first imaging device that generates an image to be stored in the storage unit based on the second signal output from the second imaging device;
A first imaging condition of the first imaging device that generates the first signal serving as a basis of an image displayed on the display unit, and the second imaging device that generates the second signal used by the imaging condition setting unit A control unit that controls the second imaging condition so that a change in the first imaging condition is smaller than a change in the second imaging condition.
An imaging device having: The imaging device according to claim 1,
The imaging device, wherein the control unit controls the frequency of change of the first imaging condition to be smaller than the frequency of change of the second imaging condition. In the imaging device according to claim 1 or 2,
The imaging apparatus, wherein the control unit controls the change width of the first imaging condition to be smaller than the change width of the second imaging condition. The imaging device according to any one of claims 1 to 3,
The imaging apparatus, wherein the first imaging condition and the second imaging condition controlled by the control unit are imaging times of the first imaging device and the second imaging device. In the imaging device according to any one of claims 1 to 4,
The imaging device, wherein the first imaging condition and the second imaging condition controlled by the control unit are gains set by the first imaging device and the second imaging device. The imaging device according to any one of claims 1 to 5,
The imaging condition setting unit generates an image to be stored in the storage unit based on a result of detecting a specific subject among the subjects based on the second signal output from the second imaging device. An imaging device for setting an imaging condition of a first imaging device. The imaging device according to claim 6,
The imaging condition setting unit generates an image to be stored in the storage unit based on a result of tracking a specific subject among the subjects based on the second signal output from the second imaging device. An imaging device for setting an imaging condition of the first imaging element to be performed. The imaging device according to any one of claims 1 to 7,
The imaging apparatus, wherein the imaging condition setting unit performs focus control of an optical system that forms an image of the subject based on the second signal output from the second imaging device. The imaging device according to any one of claims 1 to 8,
The imaging device, wherein the second imaging element includes a focus detection pixel that outputs the second signal for performing focus control, and an image pixel that outputs the second signal for generating an image. The imaging device according to claim 9,
The control unit is configured to perform imaging of the second image sensor by performing imaging to output the second signal for performing focus control and imaging to output the second signal to generate an image. An imaging device that varies the time. In the imaging device according to any one of claims 1 to 10,
The image pickup device, wherein the second image pickup device includes a pixel having a color filter transmitting light of a predetermined color and a pixel not having the color filter. The imaging device according to any one of claims 1 to 11,
The imaging apparatus, wherein the imaging condition setting unit controls an aperture value of an optical system that forms an image of the subject based on the second signal output from the second imaging device.

Images