KR20170061075A - Image sensor and image capturing apparatus - Google Patents

Abstract

An image pickup device including an image pickup unit including a plurality of photoelectric conversion elements includes an input unit for inputting image data from the outside of the image pickup device and an image processing unit for performing image processing on image data obtained by the image pickup unit and image data input from the input unit And an output unit for externally outputting the image data obtained through the image processing by the image processing unit.

Description

[0001] IMAGE SENSOR AND IMAGE CAPTURING APPARATUS [0002]

The present invention relates to an image pickup device and an image pickup apparatus.

2. Description of the Related Art In recent years, an imaging device used in an imaging device such as a digital still camera and a digital video camera has been made more multifunctional. As means for achieving the multifunctional function of the image pickup element, there is provided an image pickup section constituted by a pixel section having a photoelectric conversion element for converting light into electricity, and an image processing section for performing image processing on the image pickup signal output from the image pickup section An image pickup device has been proposed.

Japanese Patent Application Laid-Open No. 2006-49361 discloses a technique relating to an image pickup device manufactured by laminating an image pickup device chip on a signal processing chip.

The signal processing chip of the image pickup device disclosed in Japanese Patent Application Laid-Open No. 2006-49361 is provided for performing signal processing on the output from the image pickup device chip included in the semiconductor module. Therefore, in order to execute various image processes again on the image data output from the image pickup device, it is necessary to provide a separate image processing chip outside the image pickup device, and an extra component is required.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and it is an object of the present invention to enable image processing in the imaging device through inputting image data from the outside of the imaging device.

According to the present invention, there is provided an image pickup device comprising an image pickup unit including a plurality of photoelectric conversion elements, the image pickup apparatus comprising: an input unit for inputting image data from outside the image pickup device; image data obtained by the image pickup unit; And an output unit for outputting image data obtained through image processing by the image processing unit to the outside.

Furthermore, according to the present invention, there is provided an image pickup device comprising an image pickup section including a plurality of photoelectric conversion elements, the image pickup device comprising: an input section for inputting image data from outside the image pickup device; image data obtained by the image pickup section; An image processing apparatus comprising: an image processing unit for executing image processing on image data; an image pickup element having an output unit for outputting image data obtained through image processing by the image processing unit to the outside; There is provided an image pickup apparatus having a processing section and a recording processing section.

Furthermore, according to the present invention, there is provided a photoelectric conversion device comprising a first image pickup device having an image pickup section having a plurality of photoelectric conversion elements, and a second image pickup device having an image pickup section having a plurality of photoelectric conversion elements, An image processing section for executing image processing on image data input from the input section; and an output section for outputting image data obtained through image processing by the image processing section to the outside And the image processing section performs image processing on the image data input from the second image pickup element to the input section of the first image pickup device.

Further features of the present invention will become apparent from the following detailed description of the embodiments (given with reference to the accompanying drawings).

The accompanying drawings, which are incorporated in and form a part of the specification, illustrate exemplary embodiments, features and aspects of the invention and, together with the description, serve to explain the principles of the invention.
1 is a block diagram showing the configuration of an image pickup device according to a first embodiment of the present invention.
2 is a block diagram showing a configuration of an image processing section of an image pickup device according to the first embodiment.
3 is a block diagram showing a schematic configuration of an image pickup apparatus according to the first embodiment.
Fig. 4 is a flowchart showing a process executed at the time of normal photographing according to the first embodiment.
5 is a flowchart showing a process executed when an image is input in the first embodiment.
Fig. 6 is a flowchart showing image composition processing for a captured image and an input image according to the first embodiment.
7 is a block diagram showing a schematic configuration of an image pickup apparatus according to the second embodiment.
8 is a block diagram showing a configuration of a second image pickup device according to the second embodiment.
Fig. 9 is a flowchart showing an image capturing process executed when the first image capturing device performs image processing on an image captured by the second image capturing device according to the second embodiment. Fig.
10 is a flowchart showing an imaging process performed when the first imaging device captures a moving image and the second imaging device captures a still image according to the second embodiment.
Fig. 11 is a flowchart showing an image capturing process executed when the first and second image capturing devices capture a still image and a captured still image is synthesized according to the second embodiment. Fig.
Fig. 12 is a flowchart showing an image capturing process executed when first and second image capturing devices capture a moving image and a captured moving image is synthesized according to the second embodiment. Fig.
Fig. 13 is a flowchart showing an image capturing process executed when the second image capturing device is used as an auxiliary means for capturing by the first image capturing device according to the second embodiment. Fig.
14 is a block diagram showing a configuration of an image pickup device according to the third embodiment.
15 is a block diagram showing a schematic configuration of a first substrate of an image pickup device according to the third embodiment.
16 is a block diagram showing a configuration of an image processing section of an image pickup device according to the third embodiment.

Hereinafter, embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The dimensions, the formation and the relative positions of the components shown in these embodiments are appropriately changed according to various situations and the structure of the apparatus modified in accordance with the present invention, and the present invention is not limited to these embodiments.

≪ First Embodiment >

First, an image pickup device according to a first embodiment of the present invention will be described. The image pickup device according to the first embodiment has a function of receiving an image signal input from the outside of the image pickup device and executing image processing in the image pickup device with respect to the input image signal.

Hereinafter, the configuration of the image pickup device according to the first embodiment will be described in detail with reference to Figs. 1 and 2. Fig. 1 is a block diagram showing the configuration of an image pickup device according to a first embodiment of the present invention.

1, the image pickup element 302 has a structure in which a first substrate 101 (image pickup section) and a second substrate 102 (processing section) are laminated. The first substrate 101 and the second substrate 102 are bonded to each other by a micro bump or a metal film between the connection portion 106 included in the first substrate 101 and the connection portion 107 included in the second substrate 102. [ Vias, etc. to establish an electrical connection therebetween. The control circuit 103 included in the first substrate 101 and the image processing unit 108 included in the second substrate 102 are also electrically connected so as to be able to communicate with each other.

The first substrate 101 includes a light receiving section 104, an analog / digital (AD) conversion section 105, a connection section 106 and a control circuit 103. In the light receiving section 104, a plurality of pixels including a photoelectric conversion element for converting light into electricity are arranged two-dimensionally in the row direction and the column direction. The AD conversion unit 105 converts the analog image signal generated by the pixel of the light receiving unit 104 into a digital signal and the connection unit 106 converts the digital image signal (image data) output from the AD conversion unit 105, To the second substrate (102). The control circuit 103 controls various types of driving in the light receiving unit 104 and the A / D conversion unit 105.

The second substrate 102 includes an interface section 110 including a connection section 107, an image processing section 108, a memory section 109, an output section 111 and an input section 112. The connection unit 107 receives an image signal transmitted from the first substrate 101. The image processing unit 108 performs signal amplification, reference level adjustment, defective pixel correction processing, and correction processing on the digital image signal (image data) transmitted from the first substrate 101, the memory unit 109 and the input unit 112 And performs various image processing such as processing. In addition to the functions of executing the above-described various image processing, the image processing unit 108 uses the digital image signal (image data) obtained through photographing and the digital image signal (image data) transmitted from the input unit 112, A function of generating various composite images such as a difference image, an addition image, and an average image, and a function of combining moving image data and generating a moving image. Furthermore, the image processing section 108 has a function of calculating, for example, the luminance value of the subject based on the image data transmitted from the input section 112. [ The configuration of the image processing unit 108 will be described later in detail with reference to Fig.

The memory unit 109 temporarily stores image data from the image processing unit 108 and the input unit 112. The input unit 112 receives the image data input from the outside of the image pickup device 302 and transfers the image data to the image processing unit 108 and the memory unit 109 while the output unit 111 outputs the image data to the image processing unit 108. [ And outputs the image data output from the image pickup device 108 to the outside of the image pickup device 302. The image data is output from the interface section 110 to the outside of the image sensing element 302 and the image data is input to the common section from the outside of the image sensing element 302 to the interface section 110. In the first embodiment, . ≪ / RTI > The terminal of the interface unit 110 is connected to the input unit 112 and the terminal of the interface unit 110 is connected to the output unit 111 when the image data is output do. In the image pickup element 302, the first substrate 101 (imaging section), the second substrate 102 (processing section), and the respective components included therein constitute one package.

2 is a block diagram showing a configuration of the image processing unit 108 of the image pickup device 302 according to the first embodiment. In Fig. 2, the signal amplification section 201 amplifies the input digital image signal (image data) with a predetermined gain. The reference level adjusting unit 202 adjusts the signal level of the digital image signal (image data) to be a predetermined value in a dark state in which no light is incident on the light receiving unit 104. [ The correction unit 203 performs various correction processes such as a defective pixel correction process and a dark shading correction process on the image data.

The development processing section 204 performs development processing on the image data after performing white balance adjustment or the like on the image data. The image composition unit 205 uses the image data obtained through the photographing performed by the light receiving unit 104 and read by the AD conversion unit 105 and the image data transmitted from the input unit 112 and the memory unit 109 , A difference image, a sum image, and an average image.

The signal analyzing unit 206 analyzes the image data transmitted from the input unit 112 and the memory unit 109 and sets the signal amplification rate used by the signal amplification unit 201 based on the analysis result, And performs various drive settings in the light receiving unit 104 and the AD conversion unit 105 via the control circuit 103. [

3 is a block diagram showing a schematic configuration of an image pickup apparatus provided with an image pickup device 302 configured in the above manner. 3, an optical system 301 including a lens and a diaphragm is driven by a drive circuit 304 and converts the incident light incident through the optical system 301 into an electric signal by the image pickup element 302. [ The timing signal generating circuit 303 generates a signal for operating the image pickup element 302. [ In the present embodiment, signals generated by the timing signal generating circuit 303 include vertical and horizontal synchronizing signals, setting signals for setting various setting parameters in the image pickup element 302, and the like.

As described with reference to Fig. 1, the image pickup element 302 performs various image processing such as signal amplification, reference level adjustment, defective pixel correction processing, dark shading correction processing, and development processing on the electric signal obtained by converting incident light To generate an image signal. The image pickup device 302 also has a function of receiving the image data input from the recording circuit 305 and executing various image processes. The recording circuit 305 is a nonvolatile memory, a memory card, or a similar recording medium that records and holds image data output from the image pickup element 302, image data input to the image pickup element 302, and the like.

The control circuit 306 collectively drives and controls the entire image capturing apparatus including the image capturing element 302, the timing signal generating circuit 303 and the drive circuit 304. [ The operation circuit 307 receives a signal from the operation member provided in the image pickup apparatus and causes the control circuit 306 to reflect a command from the user. The display circuit 308 displays a photographed image, a live view image, various setting screens, and the like.

In the present embodiment, a configuration example in which the timing signal generating circuit 303 and the control circuit 306 of the image pickup apparatus are provided independently of the image pickup element 302 has been described. However, the present invention is not limited to such a configuration, and at least a part of functions of the timing signal generating circuit 303 and the control circuit 306 may be incorporated in the image pickup element 302. [

Various shooting processing and image generation processing performed by the imaging apparatus having the imaging element 302 in the first embodiment will be described in detail with reference to Figs. 4 to 6. Fig.

Fig. 4 is a flowchart showing the processing executed in the normal photographing in the first embodiment. In Fig. 4, first, initial settings of photographing conditions such as a photoconductive rate, a diaphragm value, and an exposure time are configured by an instruction given by the user via the operation circuit 307 (step S401). Subsequently, the optical system 301 and the like are controlled to perform exposure of the light receiving unit 104 of the imaging element 302 (step S402). After a predetermined accumulation time has elapsed, the AD conversion unit 105 converts the electric signal generated by the light-receiving unit 104 into a digital image signal (image data), and then reads the digital signal into the image processing unit 108 (step S403) .

Next, the signal amplifying unit 201, the reference level adjusting unit 202, the correcting unit 203 and the developing processing unit 204 perform signal amplification, reference level adjustment, defect Various kinds of image processing such as pixel correction processing, white balance adjustment, and development processing are executed (step S404). Finally, the image data outputted from the image processing section 108 is outputted from the output section 111 to the recording circuit 305 or the display circuit 308 outside the image pickup element 302 (step S405) Lt; / RTI >

Fig. 5 is a flowchart showing a process executed at the time of image input in the first embodiment. The processing executed at the time of image input explained with reference to Fig. 5 can be realized by, for example, acquiring RAW image data obtained through photographing performed in the past by the image capturing apparatus and stored in the outside, Can be used for re-developing based on desired developing parameters.

A nonvolatile memory serving as the recording circuit 305 after the image data input from the outside of the image pickup device 302 is output from the output unit 111 after various image processing is executed in the image processing unit 108, , Or similar recording medium, and the display of the image performed by the display circuit 308 are used. This image data may be used for various researches and analyzes. For example, in the field of surveillance cameras, image data may be used for character recognition and face recognition. In addition, image data may be used for detection of pedestrians, object recognition including marking, white lines, obstacles on roads and bicycles, and character recognition in the field of on-board cameras for automatic operation of automobiles and the like. At this time, the signal processing is not necessarily executed in all the areas of the input image, but may be executed in a limited area of the input image based on the result of the recognition of the object or the like. Further, only the necessary area may be cut by the input unit 112 or the like.

5, RAW image data stored in the recording circuit 305 is input to the image pickup element 302 via the input unit 112 (step S501). The development processing section 204 of the image processing section 108 then performs white balance adjustment, development processing, and the like on the image data input to the image pickup device 302, in accordance with an instruction issued by the user via the operation circuit 307 (Step S502). At this time, the input image data to be subjected to the image processing is subjected to signal amplification, reference level adjustment, defects in the signal amplification section 201, reference level adjustment section 202 and correction section 203 of the image processing section 108, Pixel correction processing, dark shading correction processing, and the like are executed. When image data is output from the image pickup element 302 and stored in the recording circuit 305 after these image processing, such image processing can be omitted when the image data is input to the image pickup element 302 again .

The image data output from the image processing unit 108 after the various image processes are executed is output from the output unit 111 to the recording circuit 305 or the display circuit 308 outside the image pickup device 302 (Step S503), and this process is terminated.

At this time, the processing described with reference to the flowchart of Fig. 4, that is, the timing of the image processing executed by the image processing section 108 for the photographing processing and the photographed image data, the image processing described with reference to the flowchart of Fig. 5, It is preferable that the timing of the image processing executed by the image processing section 108 for the data is different. The processing load of the image processing unit 108 can be reduced and the maximum power consumption can be suppressed by dispersing the image processing of the shot image data and the image processing of the input image data without simultaneously executing the image processing unit 108 .

Fig. 6 is a flowchart showing image synthesis processing for a captured image and an input image in the first embodiment. The image synthesis processing of the captured image and the input image, which will be described with reference to Fig. 6, can be performed, for example, in the case where an image captured in the past and stored outside the image sensing element 302, It can be used when synthesizing an image or the like.

In Fig. 6, an initial setting of photographing conditions such as a photographic speed, a diaphragm value, and an exposure time is first made by an instruction given by the user via the operation circuit 307 (step S601). Subsequently, the image data stored in the recording circuit 305 is input into the image pickup element 302 via the input unit 112, and then stored in the memory unit 109 (step S602). Since the input image data is subjected to composition processing together with the picked-up image data which is subsequently picked up and read by the image processing section 108, the input image data must be in the same state as the picked-up image data.

Concretely, since the photographed image data before synthesis which is read into the image processing section 108 is RAW image data which is not subjected to compression processing or the like, the input image data also needs to be in the same state. The photographed image data and the input image data are subjected to signal amplification, reference level adjustment, and reference level adjustment by the signal amplification unit 201, the reference level adjustment unit 202 and the correction unit 203 of the image processing unit 108, Defective pixel correction processing, dark shading correction processing, and the like. For this reason, it is preferable that similar processing of the input image data is performed at the time of photographing and then outputted and stored.

Thereafter, the optical system 301 and the like are controlled to perform exposure of the light receiving unit 104 of the imaging element 302 (step S603). After a predetermined accumulation time has elapsed, the AD conversion unit 105 converts the electric signal generated by the light receiving unit 104 into a digital image signal, and thereafter, is read to the image processing unit 108 as photographed image data (step S604) .

Subsequently, the image synthesis section 205 of the image processing section 108 synthesizes the photographed image data and the input image data stored in the memory section 109 (step S605). At this time, in order to match the photographing conditions of the input image data such as the accumulation time or the light-sensitive speed setting with the photographing condition of the photographed image data, a process of amplifying the input image data using an appropriate gain is simultaneously executed.

Then, the development processing section 204 of the image processing section 108 executes various image processing such as white balance adjustment, development processing, etc. on the synthesized image data (step S606). Finally, the image data outputted from the image processing section 108 is outputted from the output section 111 to the recording circuit 305 or the display circuit 308 outside the image pickup element 302 (step S607) Lt; / RTI >

As described above, according to the first embodiment, the image pickup device is provided with the input section for receiving the input image data, and various image processes can be performed on the image data input by the image processing section included in the image pickup device. This makes it possible to perform processing using a plurality of images such as image re-development output from the image pickup element, images captured and stored in the past, and images synthesized with newly captured images without using an external image processing circuit .

The imaging apparatus according to the first embodiment has been described above with reference to Figs. 1 to 6. However, the present invention is not limited to this and can take various forms.

For example, in the processing executed at the time of image input described with reference to Figs. 5 and 6, an image photographed by the image pickup apparatus is used as input image data. However, the present invention is not limited to this, and an image photographed and output by another imaging device can be input.

In the image combining process for the captured image and the input image described with reference to Fig. 6, a simple combined image is generated from the captured image data and the input image data as an example of the image combining process. However, the present invention is not limited to this. For example, as in the high dynamic range synthesis processing (HDR synthesis processing), the photographed data and the input image data are amplified using a predetermined gain, May be executed.

The difference between the photographing conditions of the input image and the photographing conditions of the photographing image is not limited to adjusting the input image data by performing the process of amplifying the input image data by using an appropriate gain. For example, A plurality of types of data may be input and stored in the memory unit 109 and appropriate input image data may be selected and used according to the shooting conditions used at the time of shooting each time shooting is performed. Further, in order to reduce the random noise, processing for generating average image data for the captured image data and the input image data may be performed.

For example, in the image synthesis processing for the captured image and the input image, a difference image representing the difference between the captured image data and the input image data may be generated. For example, when applying this method to a vehicle-mounted camera or the like, an obstacle on the road, a bicycle, and a pedestrian may be detected by taking a difference between the two images. This method is a method in which black image data obtained through photographing in the past and stored outside the image pickup element 302 are used for image data obtained through photographing performed by the image pickup element 302, Or in the case where the image processing section 108 of the element 302 performs the black extraction processing. For example, in the case of the black extraction processing at the time of shooting the celestial object, the difference image can be generated by subtracting the black image which is the input image data from the celestial object captured image which is the captured image data.

In the above description, an image signal pertaining to one kind of image signal is input to generate a differential image, but the present invention is not limited thereto. For example, black image data using a plurality of different photographing conditions may be input and stored in the memory unit 109, and appropriate black image data may be selected and used according to the photographing conditions used at the time of photographing do.

≪ Second Embodiment >

Next, a second embodiment of the present invention will be described. 7 is a block diagram showing a schematic configuration of an image pickup apparatus according to the second embodiment. As shown in Fig. 7, the imaging apparatus according to the second embodiment is implemented by adding an optical system 801 and a second imaging element 802 to the imaging apparatus shown in Fig. Since the other components are similar to those shown in Fig. 3, the description thereof is omitted. The imaging device 302 is referred to as a first imaging device 302 in the second embodiment in order to distinguish it from the second imaging device 802 .

8 is a block diagram showing a configuration of a second image pickup device according to the second embodiment. 8, the second image sensing element 802 includes a light receiving section 702, an analog / digital (AD) conversion section 703, an output section 704 and a control circuit 701. [ In the light receiving section 702, a plurality of pixels including a photoelectric conversion element for converting light into electricity are arranged two-dimensionally in the row direction and the column direction. The A / D converter 703 converts the analog image signal generated by the pixel of the light receiving unit 702 into a digital signal (image data), and the output unit 704 outputs the digital image And outputs a signal (image data) to the outside of the image pickup device 802. The control circuit 701 controls various types of driving in the light receiving unit 702 and the AD conversion unit 703.

Unlike the first image pickup device 302 shown in Fig. 1, the second image pickup device 802 does not have an image processing section. Therefore, in the second embodiment, incident light incident through an optical system 801 including a lens and diaphragm is converted into an electric signal by the second imaging device 802. And outputs this electric signal to the first image pickup element 302. [ Then, the image processing section 108 of the first image pickup device 302 performs various image processes.

In the second embodiment, the timing signal generating circuit 303 generates signals for operating the first imaging device 302 and the second imaging device 802, and the driving circuit 304 drives the optical system 301 and the optical system 801 . In the present embodiment, signals generated by the timing signal generating circuit 303 include vertical and horizontal synchronizing signals, setting signals for setting various setting parameters in the image pickup devices 302 and 802, and the like. The signal generated by the timing signal generating circuit 303 may include, for example, address information for specifying a transmission destination of the signal. By including the address information specifying the transmission destination in the thus generated signal, it becomes possible to commonly use the signal line. In Fig. 7, the first imaging device 302 and the second imaging device 802 are connected in parallel to the timing signal generating circuit 303 and the control circuit 306. Fig. However, the present invention is not limited to these configurations, and may be connected in series. In this case, the signal or the like inputted from the timing signal generating circuit 303 is supplied to the second image sensing element 802 via the first image sensing element 302. [ With this configuration, it is possible to simplify the control connection circuit when connecting two or more image pickup devices.

With reference to Figs. 9 to 13, various shooting processing and image generation processing performed by the imaging apparatus in the second embodiment will be described in detail. Fig. In the second embodiment, the processing that is performed only at the time of normal photographing, which requires only the first image pickup element 302, is the same as that according to the first embodiment described with reference to Fig. 4, and therefore, description thereof will be omitted.

9 shows an example in which the image data output from the second image pickup device 802 is input to the first image pickup device 302 and the first image pickup device 302 executes various image processes on the input image data, Fig. 18 is a flowchart showing an imaging process executed when data is output.

9, an initial setting of photographing conditions such as a photographing speed, a diaphragm value, and an exposure time is configured for the second image sensing device 802 according to an instruction or the like issued by the user via the operation circuit 307 (Step S901). Subsequently, the optical system 801 and the like are controlled to perform exposure of the light receiving unit 702 of the second image pickup device 802 (step S902). After the elapse of the predetermined accumulation time, the electric signal generated by the light receiving section 702 is converted from the output section 704 to the digital image signal (image data) by the AD conversion section 703, And outputted to the outside (step S903). At this time, since the second image pickup device 802 does not include the image processing section, various image correction and the like are not executed on the output image data.

The image data output from the second image sensing device 802 is input to the first image sensing device 302 (step S904). The signal amplifying unit 201 of the image processing unit 108, the reference level adjusting unit 202, the correcting unit 203 and the developing processing unit 204 are connected to the first image pickup device 302, Signal amplification, reference level adjustment, defective pixel correction processing, white balance adjustment, and development processing (step S905). Finally, the image data outputted from the image processing section 108 is outputted from the output section 111 to the recording circuit 305 or the display circuit 308 outside the first image pickup device 302 (step S906) The photographing process is terminated.

Fig. 10 is a flowchart showing an image capturing process executed when the first image capturing device 302 captures a still image and the second image capturing device 802 captures a moving image in the second embodiment. Here, as an example, a case where the first imaging device 302 captures a still image while the second imaging device 802 captures a live view image or a moving image will be described. Alternatively, while the live view image or the moving image is being taken by the first image pickup element 302, the second image pickup element 802 may photograph the still image.

10, an initial setting of photographing conditions such as a photographing speed, a diaphragm value, and an exposure time is configured for the second image sensing device 802 according to an instruction or the like issued by the user via the operation circuit 307 (Step S1001). Subsequently, the optical system 801 and the like are controlled to perform exposure of the light receiving section 702 of the second image pickup device 802, and live view photographing or moving image photographing is started (step S1002).

The AD conversion unit 703 converts an electric signal for each frame generated by the light receiving unit 702 into a digital image signal (image data), and outputs the digital image signal (image data) from the output unit 704 to the outside of the second image sensing device 802 Step S1003). At this time, since the second image pickup device 802 does not have an image processing section, various image correction and the like are not executed on the output image data.

The image data output from the second image sensing device 802 is input to the first image sensing device 302 (step S1004). The signal amplifying unit 201 of the image processing unit 108, the reference level adjusting unit 202, the correcting unit 203 and the developing processing unit 204 are connected to the first image pickup device 302, Signal amplification, reference level adjustment, defective pixel correction processing, white balance adjustment, and development processing (step S1005). The image data output from the image processing unit 108 is supplied from the output unit 111 to the recording circuit 305 and the display circuit 308 outside the first image pickup element 302 at the time of moving image capture and live view image capture, Respectively (step S1006).

If the user gives an instruction to shoot a still image via the operation circuit 307 (YES in step S1007) during live view shooting or moving image shooting, the first imaging device 302 starts shooting the still image.

First, the initial setting of the shooting conditions such as the speed of light exposure, the aperture value, and the exposure time is configured for the first image sensing element 302 in accordance with an instruction or the like issued by the user via the operation circuit 307 (step S1008). Subsequently, the optical system 301 and the like are controlled to perform exposure of the light receiving unit 104 of the first imaging device 302 (step S1009). After a predetermined accumulation time elapses, the AD conversion unit 105 converts the electric signal generated by the light receiving unit 104 into a digital image signal (image data), and thereafter, is read to the image processing unit 108 (step S1010).

The signal amplifying unit 201, the reference level adjusting unit 202, the correcting unit 203 and the developing processing unit 204 perform signal amplification, reference level adjustment, Various kinds of image processing such as pixel correction processing, white balance adjustment, and development processing are executed (step S1011). The image signal output from the image processing unit 108 is output from the output unit 111 to the recording circuit 305 or the display circuit 308 outside the first imaging device 302 (step S1012).

After the still image shooting, or when the instruction to shoot still image is not made (NO in step S1007), it is determined whether to continue live view shooting or moving image shooting (step S1013). When the live view photographing or moving image photographing is continued, a series of photographing operations return to step S1001 to perform live view photographing or moving image photographing for the next frame, and when live view photographing or moving image photographing is not continued, And ends the series of photographing operations.

11 is a flowchart showing an image capturing process executed when a still image captured by the first image capturing device 302 and the second image capturing device 802 are respectively synthesized in the second embodiment. This imaging method can be used, for example, when a plurality of parallax images for 3D display are generated using two imaging elements. When this method is used in a vehicle-mounted camera or the like, the object may be positioned on the basis of a parallax image obtained by using two imaging elements. That is, the distance from the parallax image obtained from the two imaging elements to the object can be measured using the principle of triangulation.

11, first of all, with respect to the first imaging device 302 and the second imaging device 802, the imaging speed, the aperture value, the exposure time, and the like The initial setting of the shooting conditions is configured (step S1101). Subsequently, the optical systems 301 and 801 are controlled to perform exposure of the light receiving unit 104 of the first imaging device 302 and the light receiving unit 702 of the second imaging device 802 (step S1102). After a predetermined accumulation time has elapsed, the AD conversion unit 105 converts the electric signal generated by the light receiving unit 104 of the first image pickup device 302 into a digital image signal (image data), and thereafter, (Step S1103). The AD conversion unit 703 converts an electric signal generated by the light receiving unit 702 of the second image pickup device 802 into a digital image signal (Step S1103), and is input to the image sensing element 302 (step S1104). At this time, since the image pickup device 802 does not have an image processing section, various image correction and the like are not executed on the image data output from the second image pickup device 802 and input to the image pickup device 302. [

Thereafter, the image data obtained through the imaging performed by the first imaging device 302 and the image data obtained through the imaging performed by the second imaging device 802 are supplied to the signal amplifying unit 201, the reference level adjusting unit 202, the correcting unit 203 and the developing processing unit 204 execute various image processes such as signal amplification, reference level adjustment, defective pixel correcting process, and the like. After executing various image processes, the image data pair is synthesized by the image synthesis unit 205 of the image processing unit 108 (step S1105). Next, with respect to the synthesized image data, the development processing section 204 of the image processing section 108 executes various image processing such as white balance adjustment and development processing (step S1106).

Finally, the image data output from the image processing unit 108 is outputted from the output unit 111 to the recording circuit 305 or the display circuit 308 outside the image pickup element 302 (step S1107), and the image pickup processing is terminated do.

Fig. 12 is a flowchart showing an imaging process performed when a moving image of the same subject is captured and synthesized by the first imaging device 302 and the second imaging device 802 in the second embodiment. This imaging method can be used, for example, in the case of generating a moving image at a high frame rate by alternately reading image data output from one imaging element and image data output from another imaging element and synthesizing them .

12, first of all, with respect to the first image sensing element 302 and the second image sensing element 802, an exposure speed, an aperture value, an exposure time, and the like The initial setting of the shooting conditions is configured (step S1201). Subsequently, the optical systems 301 and 801 and the like are controlled to expose the light receiving unit 104 of the first image pickup device 302 and the light receiving unit 702 of the second image pickup device 802 to start moving image pickup (step S1202) . At this time, the first image sensing element 302 and the second image sensing element 802 perform moving image sensing with a phase shift corresponding to the same frame rate and half period.

When the light receiving unit 104 of the first image sensing element 302 completes the accumulation corresponding to one frame, the AD conversion unit 105 converts the electric signal generated by the light receiving unit 104 into a digital image signal (image data) Thereafter, the image data is read to the image processing unit 108 as photographed image data (step S1203). The signal amplifying unit 201 of the image processing unit 108, the reference level adjusting unit 202, the correcting unit 203, and the developing processing unit 204 perform signal amplification and correction on the image data read to the image processing unit 108, Various kinds of image processing such as reference level adjustment, defective pixel correction processing, white balance adjustment, and development processing are executed (step S1204). The image data subjected to the signal processing is recorded in the memory unit 109 (step S1205).

Subsequently, when the light receiving section 702 of the second image capturing device 802 completes the accumulation corresponding to one frame, the AD conversion section 703 converts the electric signal generated by the light receiving section 702 into a digital image signal, From the output unit 704 to the outside of the second image sensing device 802 (step S1206). At this time, since the second image pickup device 802 does not have an image processing section, various image correction and the like are not executed on the output image data.

The image data output from the second image sensing element 802 is input to the first image sensing element 302 (step S1207). The signal amplifying unit 201 of the image processing unit 108, the reference level adjusting unit 202, the correcting unit 203, and the image processing unit 106 are connected to the image data input from the second image sensing device 802 via the input unit 112, The processing unit 204 performs various image processing such as signal amplification, reference level adjustment, defective pixel correction processing, inter-frame noise reduction processing, white balance adjustment, and development processing (step S1208). The image data subjected to the signal processing is recorded in the memory unit 109 following the image data of the first image pickup element 302 recorded before (step S1209).

Then, it is determined whether to continue moving image capture (step S1210). When moving image shooting is to be continued, a series of photographing operations are returned to step S1203, and the above-described processing is repeated. When the moving image capturing is not to be continued, the exposure of the light receiving unit 104 of the first image capturing device 302 and the light receiving unit 702 of the second image capturing device 802 is stopped and the moving image capturing is ended (step S1211) . The moving image data recorded in the memory unit 109 is output from the output unit 111 to the recording circuit 305 outside the first image sensing element 302 (step S1212), and a series of image sensing operations ends.

13 is a flowchart showing a photographing process performed when the second image pickup device 802 is used as an auxiliary means for determining the photographing condition of photographing performed by the first image pickup device 302 in the second embodiment All. Such processing is performed by, for example, based on the output signal from the second image sensing element 802, the image processing section 108 included in the first image sensing element 302 calculates the photometric value of the subject, And can be used when the photographing condition of the photographing performed by the first imaging device 302 is specified on the basis of one photometry value.

In Fig. 13, first, the initial setting of the measurement conditions such as the light-sensitive speed and the exposure time is configured for the second image sensing element 802 (step S1301). Subsequently, the optical system 801 and the like are controlled to perform exposure of the light receiving section 702 of the second image sensing element 802 to start photographing (step S1302). The AD conversion unit 703 converts an electric signal for each frame generated by the light receiving unit 702 into a digital image signal (image data), and then outputs the electric signal from the output unit 704 to the outside of the second image sensing device 802 Step S1303). At this time, since the second image pickup device 802 does not include the image processing section, various image correction and the like are not executed on the output image data.

The image data output from the second image sensing device 802 is input to the first image sensing device 302 (step S1304). Subsequently, the signal analyzing unit 206 of the image processing unit 108 analyzes the image data from the second image pickup device 802 and calculates the luminance value of the subject, for example (step S1305). Then, on the basis of the calculation result, photographing conditions for photographing performed by the first imaging device 302 such as the photospecting speed, the aperture value, and the exposure time are set (step S1306). Subsequently, the optical system 301 and the like are controlled to perform exposure of the light receiving unit 104 of the first imaging device 302 (step S1307). After a predetermined accumulation time elapses, the AD conversion unit 105 converts the electric signal generated by the light receiving unit 104 into a digital image signal (image data), and thereafter, is read to the image processing unit 108 (step S1308).

The signal amplification section 201, the reference level adjustment section 202, the correction section 203 and the development processing section 204 perform signal amplification, reference level adjustment, Correction processing, white balance adjustment, and development processing (step S1309). Finally, the image data outputted from the image processing section 108 is outputted from the output section 111 to the recording circuit 305 or the display circuit 308 outside the first image pickup device 302 (step S1310) The processing is terminated.

As described above, according to the second embodiment, since the image pickup device other than the image pickup device described in the first embodiment is provided, the image data generated through the image pickup performed by the other image pickup device is subjected to image processing Image processing can be performed using a circuit. That is, image processing for image data of two imaging elements can be performed without using an external image processing circuit.

In addition, by providing the image processing circuit inside the image pickup element, an advantage that can not be achieved when one image processing circuit provided outside the image pickup element needs to receive image data from a plurality of image pickup elements as input have. These advantages include not only speeding up but also the number of terminals required for a circuit when outputting the result of combining image data from a plurality of image pickup devices.

In the above-described embodiment, the first imaging device 302 has an image processing unit, but the second imaging device 802 has no image processing unit. However, the present embodiment is not limited to such a configuration. For example, the second imaging device 802 also includes an image processing unit and an input unit similar to the first imaging device 302, The image processing section of the other image pickup element is not operated, that is, the power supply is not received, or the image processing section of the other image pickup element is placed under the power saving control.

In the above-described embodiment, a configuration example in which the control circuit 306 of the image pickup apparatus is installed independently of the first image pickup device 302 and the second image pickup device 802 has been described. However, the present embodiment is not limited to such a configuration, and at least part of the configuration of the control circuit 306 may be built in the first imaging device 302 or the second imaging device 802. For example, when the first imaging device 302 includes the timing signal generating circuit 303 as an internal component, the second imaging device 802 supplies a signal to the second imaging device 802, 1 image pickup element 302. [0050]

One of the two imaging elements may pick up a visible ray and the other one may capture an invisible ray such as an infrared ray. Alternatively, one of the two imaging elements (for example, an imaging element that captures visible light) captures a color image, and the other one (e.g., an imaging element that captures an invisible ray) It may be photographed.

At this time, the first imaging device 302 and the second imaging device 802 may be connected to the timing signal generating circuit 303 and the control circuit 306 in series. In this case, the image data output from the second image sensing device 802 is recorded in the recording circuit 305 or the like via the first image sensing device 302. [ In such a configuration, when the image data output from the second image pickup device 802 is displayed on the display circuit 308, there is a fear that processing in the first image pickup device 302 delays the display timing. In order to reduce the delay timing, data may be controlled so as to bypass the input section 112 and the output section 111 in the first image pickup element 302. With this configuration, it is possible to reduce the delay of the recording or display timing when two or more image pickup devices are connected in series.

≪ Third Embodiment >

Next, a third embodiment of the present invention will be described. The image pickup device 1400 according to the third embodiment has a different structure from the image pickup device 302 according to the first and second embodiments. The imaging apparatus according to the third embodiment is different from the imaging apparatus described with reference to Fig. 3 in the first embodiment or the imaging apparatus explained in the second embodiment except that the imaging element 302 is replaced with an imaging element 1400 Is the same as the imaging device described with reference to Fig. Since the other structures are similar to those described in the first and second embodiments, the description is omitted.

14 is a block diagram showing a configuration of an image pickup device 1400 according to the third embodiment. In Fig. 14, the image pickup device 1400 has a structure in which a first substrate 1401 and a second substrate 1402 are laminated. The first substrate 1401 and the second substrate 1402 are electrically connected to each other by a plurality of penetrating electrodes 1406 between a connecting portion 1404 in the first substrate 1401 and a connecting portion 1405 in the second substrate 1402 At this time, the connection portion 1404 is provided in a one-to-one correspondence with the pixel sets each composed of one or more pixels and has the same number as the connection portion 1405. [ These through electrodes are, for example, through-silicon vias (TSV).

The first substrate 1401 includes a light receiving section 1411, a connection section 1404, and a control circuit 1410. In the light receiving section 1411, a plurality of pixels 1403 including photoelectric conversion elements for converting light into electricity are arranged two-dimensionally in the row direction and the column direction, and the signal generated by the light receiving section 1411 is connected to the connection section 1404 To the second substrate 1402 via the second substrate 1402. The control circuit 1410 controls various types of driving in the light receiving section 1411. [ In the description of the third embodiment, it is assumed that one connection section 1404 is provided per pixel set composed of 2X2, that is, four pixels 1403 as shown in Fig.

The second substrate 1402 includes a connecting portion 1405, an image processing portion 1408, a memory portion 109 and an interface portion 110 composed of an input portion 112 and an output portion 111. The components of the memory unit 109 and the interface unit 110, that is, the input unit 112 and the output unit 111 are similar to those described with reference to FIG. 1, and thus the description thereof is omitted here. The connection section 1405 receives an image signal transmitted from the first substrate.

The image processing unit 1408 converts the analog signal transmitted from the first substrate 1401 through the connection unit 1405 into a digital signal. The image processing unit 1408 performs various kinds of processing such as signal amplification, reference level adjustment, defective pixel correction processing, development processing, and the like on the image data obtained through AD conversion and the image data transmitted from the memory unit 109 and the input unit 112 And executes image processing. The image processing unit 1408 includes a plurality of partial image processing units 1407 provided so as to correspond one by one to the pixel sheets each composed of 2X2, that is, four pixels 1403. The configuration of the image processing unit 1408 will be described later in detail with reference to FIG.

15, the following will describe in detail the configuration of the components of the light-receiving portion 1411 of the image pickup device 1400 and peripheral components according to the third embodiment. Fig. 15 is a block diagram showing a schematic configuration of a first substrate 1401 of an image pickup device 1400 according to the third embodiment.

15, the first substrate 1401 of the image pickup device 1400 includes a light receiving portion 1411 in which a plurality of pixels 1403 are arranged in a matrix direction, a control circuit 1410, 1501). The control circuit 1410 sends drive pulses to the pixels 1403 via the drive signal line 1501. [ In this case, for simplicity, one drive signal line 1501 is provided for each row or every two rows in the figure, but actually, a plurality of drive signal lines suitable for controlling the drive of the pixel 1403 are connected to each pixel . 2X2, that is, four pixels 1403 constituting the pixel set are connected to the same connection portion 1404. [ The control circuit 1410 performs control so that the signals generated by the respective pixels 1403 sharing the same connection portion 1404 are not mixed with each other and the generated signals are sequentially output to the second substrate 1402. [ In Fig. 15, although the pixels 1403 constituting the light-receiving unit 1411 are arranged in 8 rows and 8 columns, they are actually arranged in thousands rows and thousands columns.

Here, in the first embodiment, the same components as those of the imaging element 302 described with reference to Figs. 1 and 2 are denoted by the same reference numerals, and a detailed description thereof will be omitted.

16 is a block diagram showing the configuration of the image processing section 1408 of the image pickup device 1400 according to the third embodiment. In the third embodiment, the image processing unit 1408 includes a plurality of partial image processing units 1407, a correction unit 1603, an image synthesis unit 1605, and a development processing unit 1604. In the example shown in the third embodiment, the partial image processing unit 1407 is provided in a one-to-one correspondence with pixel sets each composed of 2X2, that is, four pixels 1403, and has a connection unit 1405, an AD conversion unit 1600, A signal amplifying unit 1601, a reference level adjusting unit 1602, and a signal analyzing unit 1606. The A / D converter 1600 converts an analog image signal from each pixel 1403 read via the connection unit 1405 into a digital signal (image data). The other components are similar to those of the image processing unit 108 of the image pickup device 302 of the first embodiment described with reference to Fig. 2, and therefore, description thereof is omitted.

At this time, various correction processing in the correction unit 1603, image synthesis processing in the image composition unit 1605, and development processing in the development processing unit 1604 are performed in a plurality of partial image processing units 1407 After merging the image data obtained through various image processing into one image data.

Next, an example of the photographing process according to the third embodiment will be described in detail. Here, it is assumed that the image pickup device 1400 is used instead of the first image pickup device 302 in the image pickup apparatus having the configuration shown in Fig. 7 described in the second embodiment, and in order to distinguish it from the second image pickup device 802 , And the first image pickup device 1400. Hereinafter, with reference to the flowchart of Fig. 13, the processing executed when the second imaging device 802 is used as an auxiliary means for determining the imaging conditions of the first imaging device 1400 will be described. In this process, for example, based on the output signal from the second image pickup device 802, the image processing section 108 included in the first image pickup device 1400 calculates the photometric value of the subject, And can be used when the photographing condition for the first imaging device 1400 is determined based on the photometric value.

At this time, the processing in steps S1301 to S1304 in which the second image sensing element 802 acquires an image and inputs the output signal to the image sensing element 1400 is similar to the processing in the second embodiment, and therefore a description thereof will be omitted.

Next, the signal analyzing unit 1606 of the image processing unit 1408 analyzes the image data from the second image sensing device 802, for example, correlates the luminance value of the subject with the pixel 1403 one by one (Step S1305). Then, on the basis of the calculation result, photographing conditions for photographing performed by the first imaging device 1400 such as the photospecting speed, the aperture value, and the exposure time are set (step S1306).

At this time, in the third embodiment, on the basis of the analysis result by the signal analyzing section 1606, by using imaging conditions such as different charge accumulation times in the first imaging device 1400 according to the luminance value , It is possible to realize an operation supporting a wider dynamic range. Specifically, the pixel 1403, which is estimated to have a high luminance value of the subject, performs control so that the accumulation time becomes shorter while the imaging element 1400 is exposed via the control circuit 103. [ Conversely, the pixel 1403, which is estimated to have a low luminance value of the subject, performs control so that the charge accumulation time becomes longer. By this control, it is possible to acquire an image having a dynamic range larger than that obtained at the time of normal photographing. The amplification factor used by the signal amplification unit of the partial image processing unit 1407 is controlled for each subject luminance generated by each pixel 1403 based on the analysis result of the subject luminance by the signal analysis unit 1606 , A similar advantageous effect can be obtained.

Thereafter, the processes from the image acquisition by the first image pickup device 1400 to the image data output (steps S1307 to S1310) are the same as the process described in the second embodiment, and therefore the description is omitted.

14 to 16, the image pickup apparatus according to the third embodiment has been described. However, the present invention is not limited to this and may be implemented in various forms.

For example, according to the configuration of the imaging element 1400 described with reference to Figs. 14 and 15, 2X2, that is, four pixels 1403 constituting a pixel set are connected to a common connecting portion 1404, a connecting portion 1405, a penetrating electrode 1406, And the partial image processing unit 1407 are shared. However, the present invention is not limited to this, and more pixels 1403 may share them, or one pixel 1403 may occupy them.

For example, according to the configuration of the image processing unit 1408 described with reference to FIG. 16, each partial image processing unit 1407 includes a corresponding reference level adjusting unit 1602 and a signal analyzing unit 1606. However, the present invention is not limited to this, and the image processing section 1408 may include one reference level adjusting section and a signal analyzing section.

The imaging apparatus according to the first to third embodiments has been described with reference to Figs. 1 to 16. However, the present invention is not limited to this and may be implemented in various forms.

For example, the imaging element 302 described with reference to Fig. 1 is configured so that two substrates are stacked. However, the present invention is not limited thereto, and all the elements including the light receiving unit 104 and the image processing unit 108 may be mounted on one substrate, or three or more substrates may be stacked.

3 and 7, all of the image processing for the image data generated by the image pickup device is performed by the image processing unit 108 inside the image pickup device 302 And an image processing circuit is not provided outside the image pickup device. However, the present invention is not limited to this, and an image processing circuit may be provided separately from the image pickup device 302, and various kinds of image processing for the photographed image data may be performed by an internal image processing section 108 and an external image processing circuit You can share.

For example, according to the configuration of the image pickup device 302 described with reference to Fig. 1, the interface unit 110 can transfer image data from the interface unit 110 to the outside of the image pickup device 302 And uses a common terminal for inputting image data from the outside of the image pickup device 302 to the interface section 110. [ However, the present invention is not limited to this, and the input terminal and the output terminal may be provided separately from each other. Particularly, when image signals are frequently input to the first image pickup device 302 as in the second embodiment, it is preferable that the input terminal and the output terminal are provided separately from each other instead of using a common terminal for input and output Do.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to those embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications, equivalent structures and functions.

Claims (29)

An imaging device comprising an imaging section including a plurality of photoelectric conversion elements,
An input unit for inputting image data from the outside of the image pickup device;
An image processing unit for executing image processing on image data obtained by the image pickup unit and image data input from the input unit;
And an output unit for outputting image data obtained through image processing by the image processing unit to the outside.
The method according to claim 1,
Wherein the image pickup section, the input section, the image processing section, and the output section are provided in one package.
The method according to claim 1,
Wherein the timing of photographing by the image pickup unit is different from the timing at which image data is input from the outside of the image pickup device to the input unit.
The method according to claim 1,
Further comprising a supply section for supplying the image data obtained by the image pickup section to the image processing section, wherein the supply section is independent of the input section.
The method according to claim 1,
Wherein the image data input from the input unit is used for at least one of display, recording and analysis.
The method according to claim 1,
Wherein the image processing unit executes development processing on the image data input to the input unit.
The method according to claim 1,
Wherein the image processing unit performs a combining process of combining the image data obtained by the image pickup unit and the image data input by the input unit.
8. The method of claim 7,
In the combining processing, the image processing unit amplifies the image data obtained by the imaging unit and the image data input by the input unit using different gains.
8. The method of claim 7,
In the combining processing, the image processing unit obtains the image data obtained by the image pickup unit and the average of the image units input from the input unit.
The method according to claim 1,
Wherein the image processing unit subtracts the image data input from the input unit from the image data obtained by the image pickup unit.
The method according to claim 1,
Wherein the image processing unit is provided with a plurality of photoelectric conversion element sets each composed of a plurality of photoelectric conversion elements in one-to-one correspondence.
12. The method of claim 11,
Further comprising a plurality of connecting portions provided in a one-to-one correspondence with the set of photoelectric conversion elements each composed of a predetermined number of photoelectric conversion elements,
Wherein the image processing unit is provided in a one-to-one correspondence with the plurality of connection units.
The method according to claim 1,
Wherein the common terminal serves as both a component of the input section and a component of the output section.
The method according to claim 1,
Wherein the imaging section and the image processing section are formed on different substrates stacked and electrically connected to each other.
An image pickup device comprising an image pickup section including a plurality of photoelectric conversion elements, the image pickup device comprising: an input section for inputting image data outside the image pickup device; image processing means for performing image processing on the image data obtained by the image pickup section and the image data input from the input section And an output unit for outputting image data obtained through image processing by the image processing unit to the outside,
A display processing unit of image data output from the image pickup element,
And a recording processing unit.
A first image pickup element having an image pickup section having a plurality of photoelectric conversion elements,
And a second image pickup element having an image pickup section having a plurality of photoelectric conversion elements,
Wherein the first imaging device includes:
An input unit for inputting image data from outside,
An image processing unit for executing image processing on the image data input by the input unit,
And an output unit for externally outputting the image data obtained through the image processing by the image processing unit,
Wherein the image processing unit executes image processing on the image data input from the second image pickup element to the input unit of the first image pickup element.
17. The method of claim 16,
And a control section for controlling the first imaging device and the second imaging device.
17. The method of claim 16,
And the image pickup section, the input section, the image processing section, and the output section of the first image pickup device are provided in one package.
17. The method of claim 16,
Wherein timing of photographing by the image pickup section is different from timing when image data is input from the outside of the image pickup device to the input section.
17. The method of claim 16,
Wherein the image data input from the input unit is used for at least one of display, recording and analysis.
17. The method of claim 16,
Wherein the image processing section executes development processing on the image data input to the input section.
17. The method of claim 16,
And the image processing section executes image processing on image data obtained from the first image pickup device.
17. The method of claim 16,
Wherein the image processing section performs a combining process of combining image data obtained by the image pickup section of the first image pickup device and image data input from the input section.
17. The method of claim 16,
Wherein the first imaging element captures a still image and the second imaging element captures a moving image.
17. The method of claim 16,
Wherein the first imaging element captures a moving image and the second imaging element captures a still image.
17. The method of claim 16,
Wherein the first imaging element and the second imaging element capture the same subject having a predetermined parallax.
17. The method of claim 16,
Wherein the first imaging element and the second imaging element capture a moving image having different frame phases.
17. The method of claim 16,
Wherein the image pickup section and the image processing section of the first image pickup device are formed on different substrates stacked and electrically connected to each other.
17. The method of claim 16,
A display processing unit of the image data,
And a recording processing unit.

Images