JP2019145856A - Imaging device and method for controlling the same - Google Patents

Abstract

To provide an imaging device that can perform processing on a super-resolution subject image in the same manner as processing on a normal subject image, and a method for controlling the imaging device.SOLUTION: When the repeating interval of color filters of an image sensor 5 corresponds to the distance of N (even number) photoelectric conversion elements, an imaging device shifts the image sensor 5 to all positions obtained by multiplying a coefficient obtained by multiplying a natural number m including all 0 equal to or less than N/2 by N/(N+1) by the pitch of the photoelectric conversion elements, and positions obtained by multiplying a coefficient obtained by multiplying -m by N/(N+1) by the pitch of the photoelectric conversion elements. When the repeating interval corresponds to the distance of the odd number of photoelectric conversion elements, the imaging device also shifts the image sensor 5 to the positions obtained in the same manner. The arrangement of color components of a super-resolution subject image obtained by the shifted image sensor corresponds to the arrangement of the color filters, and processing on the super-resolution subject image can be performed in the same manner as processing on a normal subject image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus and a control method thereof.

  Many super-resolution techniques are known as methods for improving the resolution of a subject image without increasing the number of pixels of the image sensor. As an example of the super-resolution technique, there is a so-called pixel shift super-resolution technique. The pixel-shifting super-resolution technique is to capture a plurality of subjects by shifting the position of the image sensor, and to generate a single subject image by combining the obtained subject images. There is one in which a single-plate image pickup device in which a color filter is arranged is moved by a movement amount that is 1/2, 3/2, or 1 times the pixel pitch (Patent Document 1). In addition, there is an image obtained by shifting an image sensor having a Bayer array color filter nine times at a 2/3 pixel pitch (Patent Document 2). Further, there is a technique that obtains a second image that is moved with a movement amount of less than one pixel with respect to the first image by moving the correction lens (Patent Document 3).

Japanese Unexamined Patent Publication No. 2016-5094 JP 2008-306525 A JP 2012-156778

  None of Patent Documents 1 to 3 clearly describes how to shift the position of the solid-state electronic imaging device in accordance with the repetition interval of the color filter. For this reason, the pixel arrangement of the super-resolution subject image obtained when the solid-state electronic image sensor is shifted is different from the pixel arrangement of the normal subject image obtained when the solid-state electronic image sensor is not shifted. is there. If these pixel arrangements are different, the processing of the video signal representing the super-resolution subject image cannot be made the same as the processing of the video signal representing the normal subject image.

  According to the present invention, the processing of the video signal representing the super-resolution subject image is the same as the processing of the video signal representing the normal subject image, regardless of the repetition interval of the color filter. With the goal.

  In the image pickup apparatus according to the first invention, a plurality of photoelectric conversion elements are arranged in the row direction and the column direction, respectively, and a color filter having a characteristic of transmitting a specific color component is provided on the light receiving surface of the photoelectric conversion element. A solid-state electronic image sensor that is formed by being periodically repeated in the row direction and the column direction, and that converts the signal charge accumulated in the photoelectric conversion element into a video signal representing the subject image and outputs it by imaging the subject. The solid-state electronic image sensor is positioned relatively shifted in the row direction and the column direction, and the repetition interval of the color filter corresponds to the distance of N photoelectric conversion elements that are even numbers in the row direction and the column direction, respectively. All of m, which are natural numbers including 0 of N / 2 or less, are multiplied by the coefficient m · N / (N + 1) obtained by multiplying N / (N + 1) by the pitch of the photoelectric conversion element. All of m, which are natural numbers including 0 and N / 2 or less, are negative, and -m · N / (N + 1) is a coefficient obtained by multiplying N / (N + 1) by the pitch of the photoelectric conversion element. Positioning means for relatively positioning the solid-state electronic image pickup device at all the positions obtained in this manner, and an imaging control means for causing the solid-state electronic image pickup device to pick up images at positions relatively positioned by the positioning means. And

  The first invention also provides a control method suitable for the imaging apparatus. That is, in this method, a solid-state electronic image sensor has a plurality of photoelectric conversion elements arranged in the row direction and the column direction, respectively, and a color having a characteristic of transmitting a specific color component on the light receiving surface of the photoelectric conversion element. -The filter is formed by periodically repeating each in the row direction and the column direction, and the signal charge accumulated in the photoelectric conversion element is converted into a video signal representing the subject image and output by imaging the subject, The positioning means relatively positions the solid-state electronic image sensor in the row direction and the column direction, and the repetition interval of the color filter corresponds to the distance of N photoelectric conversion elements that are even numbers in the row direction and the column direction, respectively. In this case, the coefficient m · N / (N + 1) obtained by multiplying all m, which are natural numbers including 0 of N / 2 or less, by N / (N + 1), is multiplied by the pitch of the photoelectric conversion element. A coefficient obtained by multiplying all positions and all m, which are natural numbers including 0 less than or equal to N / 2, as negative, -m multiplied by N / (N + 1), and multiplying -m · N / (N + 1) by the pitch of the photoelectric conversion element. The solid-state electronic image pickup device is relatively positioned at all the positions obtained in this manner, and the image pickup control means causes the solid-state electronic image pickup device to pick up an image for each position relatively positioned by the positioning means.

  In the image pickup apparatus according to the second invention, a plurality of photoelectric conversion elements are arranged in the row direction and the column direction, respectively, and a color filter having a characteristic of transmitting a specific color component is provided on the light receiving surface of the photoelectric conversion element. A solid-state electronic image sensor that is formed by being periodically repeated in the row direction and the column direction, and that converts the signal charge accumulated in the photoelectric conversion element into a video signal representing the subject image and outputs it by imaging the subject. The solid-state electronic image sensor is positioned relatively shifted in the row direction and the column direction, and the repetition interval of the color filter corresponds to the distance of the Q photoelectric conversion elements that are odd numbers in the row direction and the column direction, respectively. In this case, the coefficient p · Q / (Q + 1) obtained by multiplying all ps, which are natural numbers including 0 of Q / 2 or less, by Q / (Q + 1), is multiplied by the pitch of the photoelectric conversion element. All the positions and all ps, which are natural numbers including 0 of Q / 2 or less, are negative. −P · Q / (Q + 1) is a coefficient obtained by multiplying p by Q / (Q + 1). Relative to all the positions obtained by multiplication, and all positions obtained by multiplying the coefficient 2p · Q / (Q + 1) or the coefficient –2p · Q / (Q + 1) by the pitch of the photoelectric conversion element, And an image pickup control means for causing the solid-state electronic image pickup device to pick up an image at each position relatively positioned by the positioning means.

  The second invention also provides a control method suitable for the imaging apparatus. In other words, in this method, a solid-state electronic image sensor has a plurality of photoelectric conversion elements arranged in the row direction and the column direction, respectively, and a color having a characteristic of transmitting a specific color component on the light receiving surface of the photoelectric conversion element. -The filter is formed by periodically repeating each in the row direction and the column direction, and the signal charge accumulated in the photoelectric conversion element is converted into a video signal representing the subject image and output by imaging the subject, The positioning means relatively positions the solid-state electronic image sensor in the row direction and the column direction, and the repetition interval of the color filter corresponds to the distance of Q photoelectric conversion elements that are odd numbers in the row direction and the column direction, respectively. In this case, the coefficient p · Q / (Q + 1) obtained by multiplying all ps, which are natural numbers including 0 of Q / 2 or less, by Q / (Q + 1), is multiplied by the pitch of the photoelectric conversion element. And the coefficient of the photoelectric conversion element multiplied by -p · Q / (Q + 1), which is obtained by multiplying -p / Q / (Q + 1) by -p / Q / (Q + 1). Relative to all the positions obtained by multiplication, and all positions obtained by multiplying the coefficient 2p · Q / (Q + 1) or the coefficient –2p · Q / (Q + 1) by the pitch of the photoelectric conversion element, The image pickup control means causes the solid-state electronic image pickup device to pick up an image at each position relatively positioned by the positioning means.

  A composite subject that generates a composite subject image signal representing a composite subject image using a plurality of video signals obtained by imaging using a solid-state electronic image sensor at each relatively positioned position under the control of the imaging control means Image signal generating means may be further provided.

  Mode setting means for setting the shifted imaging mode or the normal imaging mode, an optical low-pass filter that can be placed in front of the solid-state electronic image sensor and can be retracted from the front of the solid-state electronic image sensor, and the shifted imaging mode by the mode setting means Is set to retract the optical low pass filter from the front of the solid-state electronic image sensor, and the mode setting means sets the normal image pickup mode so that the optical low-pass filter is set in front of the solid-state electronic image sensor. When the normal imaging mode is set by the optical low-pass filter control means and the mode setting means, the relative positioning of the solid-state electronic image pickup device by the positioning means is stopped (relative to the solid-state electronic image pickup device). Position to stop positioning and position the solid-state electronic image sensor at the reference position) Because stop means may further comprise a.

  In the first invention, the aperture diameter of the light receiving surface of the photoelectric conversion element may be equal to or larger than a value obtained by multiplying the pitch of the photoelectric conversion element by 1 / (N + 1), or less than 2 / (N + 1) to the pitch of the photoelectric conversion element. A value obtained by multiplying the value may be used, or a value obtained by multiplying the pitch of the photoelectric conversion element by 2 / (N + 1). The aperture diameter of the light receiving surface of the photoelectric conversion element is a value obtained by multiplying the pitch of the photoelectric conversion element by 1 / (N + 1) or more and multiplying the pitch of the photoelectric conversion element by 2 / (N + 1) or less. Is preferred.

  In the second invention, the aperture diameter of the light receiving surface of the photoelectric conversion element may be equal to or larger than a value obtained by multiplying the pitch of the photoelectric conversion element by 1 / (Q + 1), or the pitch of the photoelectric conversion element is Q / (Q + 1). A value obtained by multiplying the following values or a value obtained by multiplying the pitch of the photoelectric conversion elements by Q / (Q + 1) may be used. The aperture diameter of the light receiving surface of the photoelectric conversion element is a value obtained by multiplying the pitch of the photoelectric conversion element by 1 / (Q + 1) or more and multiplying the pitch of the photoelectric conversion element by 2 / (Q + 1) or less. Is preferred.

  The composite subject image signal generation means represents, for example, a plurality of video signals obtained by imaging using a solid-state electronic image sensor at each relatively shifted position under the control of the imaging control means to represent a composite subject image It is a memory for storing as a composite subject image signal.

  It is preferable that the exposure time of the solid-state electronic imaging device is the same for each position relatively positioned by the positioning means.

  It further includes an amplifier circuit that amplifies each video signal of the plurality of video signals obtained by imaging using the solid-state electronic imaging device for each position relatively positioned by the positioning means with the same amplification factor. Good.

  An amplification circuit that amplifies each video signal obtained by imaging using a solid-state electronic imaging device at each position relatively positioned by the positioning means, and the video signal level after amplification by the amplification circuit is equal. As described above, an amplification control unit that controls amplification in the amplification circuit according to the exposure time of the solid-state electronic image sensor at each position relatively positioned by the positioning unit may be further provided.

  The positioning means positions, for example, a solid-state electronic image sensor.

  In the case of further including an imaging lens that is provided in front of the solid-state electronic image sensor and is movable in a direction perpendicular to the central axis of the solid-state electronic image sensor, the positioning means may position the imaging lens, for example. Good.

  In the case where a holding base for holding the imaging device is further provided so as to be movable in a direction perpendicular to the optical axis of the solid-state electronic imaging device, the positioning means may position the imaging device, for example.

  The color filter array is, for example, a Bayer array.

  Mode setting means for setting a shifted imaging mode or a normal imaging mode, a mechanical shutter disposed in front of the solid-state electronic image sensor, and performing an operation of exposing the solid-state electronic image sensor for an exposure time corresponding to the shutter speed.・ Electronic shutter means for performing an electronic shutter for outputting the signal charge accumulated during the exposure time according to the speed from the solid-state electronic image sensor as a video signal, and the mode setting means for setting the shifted imaging mode to There may be further provided a shutter control unit that performs an electronic shutter by the shutter control unit and performs an operation by a mechanical shutter when the normal imaging mode is set by the mode setting unit.

  You may further provide the mechanical shutter arrange | positioned ahead of the solid-state electronic image sensor. In this case, the image pickup control means causes the solid-state electronic image pickup device to pick up an image at each position relatively positioned by the positioning means with the mechanical shutter opened, and for each position relatively positioned by the positioning means. After imaging by the solid-state electronic image sensor, a plurality of video signals obtained by causing the solid-state electronic image sensor to capture an image with the mechanical shutter closed and an image with the solid-state electronic image sensor opened with the mechanical shutter open. The image processing apparatus may further include black level correction means for correcting the black level using a reference video signal obtained by causing the solid-state electronic image pickup device to pick up each video signal with the mechanical shutter closed.

  A shift imaging mode setting means for setting a shift imaging mode, a camera shake correction mode setting means for setting a camera shake correction mode, provided in front of the solid-state electronic image sensor, and movable in a direction perpendicular to the central axis of the solid-state electronic image sensor And a camera shake correction unit that performs camera shake correction by moving the image pickup lens in a direction perpendicular to the central axis of the solid-state electronic image sensor in response to the camera shake correction mode being set by the image pickup lens and the camera shake correction mode setting unit. Further, it may be provided. In this case, the positioning unit positions the solid-state electronic image sensor at a position relatively positioned by the positioning unit in response to the shifting imaging mode being set by the shifting imaging mode setting unit.

  According to the first invention, the color filter is periodically and repeatedly formed in the row direction and the column direction on the light receiving surface of the photoelectric conversion element of the solid-state electronic image sensor. When the repetition interval of the color filter corresponds to the distance between N photoelectric conversion elements that are even numbers in the row direction and the column direction, all m that are natural numbers including 0 of N / 2 or less are used. All the positions obtained by multiplying the coefficient m · N / (N + 1) multiplied by N / (N + 1) by the pitch of the photoelectric conversion elements and all m which are natural numbers including 0 of N / 2 or less are negative − The solid-state electronic image sensor is relatively positioned at all positions obtained by multiplying the coefficient -m · N / (N + 1) by m multiplied by N / (N + 1) by the pitch of the photoelectric conversion element. When a composite subject image signal representing a composite subject image is generated using a plurality of video signals obtained by imaging using a solid-state electronic image sensor at each relatively positioned position, the generated composite subject image The pixel array of the composite subject image represented by the signal is the same as the pixel array of the normal subject image obtained when the solid-state electronic image sensor is not relatively positioned, and the processing of the composite subject image signal is performed in the normal subject This can be done in the same way as the processing of a video signal representing an image.

  Also in the second invention, color filters are periodically and repeatedly formed in the row direction and the column direction on the light receiving surface of the photoelectric conversion element of the solid-state electronic image sensor. When the repetition interval of the color filter corresponds to the distance between Q photoelectric conversion elements that are odd numbers in the row direction and the column direction, all the ps that are natural numbers including 0 of Q / 2 or less are used. All the positions obtained by multiplying the coefficient p · Q / (Q + 1) multiplied by Q / (Q + 1) by the pitch of the photoelectric conversion elements and all ps which are natural numbers including 0 of Q / 2 or less are negative − Coefficients obtained by multiplying p by Q / (Q + 1) −p · Q / (Q + 1) by the pitch of the photoelectric conversion element, coefficients 2p · Q / (Q + 1) or coefficients −2p · Q / The solid-state electronic image sensor is relatively positioned at a position obtained by multiplying (Q + 1) by the pitch of the photoelectric conversion element. When a composite subject image signal representing a composite subject image is generated using a plurality of video signals obtained by imaging using a solid-state electronic image sensor at each relatively positioned position, the generated composite subject image The pixel array of the composite subject image represented by the signal is the same as the pixel array of the normal subject image obtained when the solid-state electronic image sensor is not relatively positioned, and the processing of the composite subject image signal is performed in the normal subject This can be done in the same way as the processing of a video signal representing an image.

  In this way, the processing of the video signal representing the super-resolution subject image is the same as the processing of the video signal representing the normal subject image, regardless of the repetition interval of the color filter. it can. The relative positioning of the solid-state electronic image pickup device may be performed by moving the solid-state electronic image pickup device itself, or the solid-state electronic image pickup device is moved and positioned relatively by moving an imaging lens, an imaging device, or the like. May be. In this specification, natural numbers include 0 (a positive integer).

It is a block diagram which shows the electric constitution of a digital camera. It is an example of the color filter currently formed in the CMOS image sensor. It is an example of the shift position of a CMOS image sensor. It is an example of the shift position of a CMOS image sensor. It is an example of the shift position of a CMOS image sensor. It is an example of the shift position of a CMOS image sensor. A part of the composite subject image is shown. It is an example of the color filter currently formed in the CMOS image sensor. It is an example of the shift position of a CMOS image sensor. It is an example of the shift position of a CMOS image sensor. It is an example of the shift position of a CMOS image sensor. It is an example of the shift position of a CMOS image sensor. It is an example of the shift position of a CMOS image sensor. A part of the composite subject image is shown. It is an example of the processing procedure of a digital camera. It is an example of the opening of a CMOS image sensor. It is an example of the opening of a CMOS image sensor. The position of the opening when the CMOS image sensor is shifted is shown. It is an example of the opening of a CMOS image sensor. The position of the opening when the CMOS image sensor is shifted is shown. It shows how digital image data is stored in memory. It is a flowchart which shows the process sequence of a digital camera. It is a flowchart which shows the process sequence of a digital camera. It is a block diagram which shows the electric constitution of a digital camera. It is a block diagram which shows the electric constitution of a digital camera. A Bayer array is shown. It is a flowchart which shows the process sequence of a digital camera. It is a flowchart which shows the process sequence of a digital camera. It is a flowchart which shows the process sequence of a digital camera. It is an example of the opening of a CMOS image sensor. It is an example of the opening of a CMOS image sensor. The relationship between the Nyquist frequency and the spatial frequency of the subject and the response is shown.

  FIG. 1 shows an embodiment of the present invention and is a block diagram showing an electrical configuration of a digital camera 1 (imaging device).

  The overall operation of the digital camera 1 is controlled by a CPU (central processing unit) 11.

  The digital camera 1 includes a shutter release button, buttons such as a mode setting switch, and an operation device 7 including switches. An output signal from the operating device 7 is input to the CPU 11.

  The digital camera 1 includes a complementary metal-oxide-semiconductor (CMOS) image sensor 5 (solid-state electronic image sensor). In this embodiment, the CMOS image sensor 5 is used, but other solid-state electronic image sensors such as a CCD (Charge-Coupled Device) image sensor may be used. A lens apparatus 2 including an imaging lens, a focus lens, a zoom lens, and the like is provided in front of the CMOS image sensor 5 (subject side). A mechanical shutter 3 is disposed between the lens device 2 and the CMOS image sensor 5. The mechanical shutter 3 performs an operation of exposing a photoelectric conversion element included in the CMOS image sensor 5 for an exposure time corresponding to the shutter speed. An optical low pass filter 4 is disposed between the mechanical shutter 3 and the CMOS image sensor 5. The position of the lens included in the lens device 2, opening / closing of the mechanical shutter 3, transfer of signal charges accumulated in the CMOS image sensor 5, and the like are controlled by the device control device 12. The optical low-pass filter 4 advances onto the optical path of the light beam representing the subject image (can be placed in front of the CMOS image sensor 5) and can be retracted from the optical path (can be retracted from the front of the CMOS image sensor 5). These devices can be advanced and retracted by the device control device 12. When the optical low-pass filter 4 advances on the optical path, the light beam representing the subject image is denoised by the optical low-pass filter 4, and the subject image from which the noise has been removed is the CMOS image sensor 5. An image is formed on the light receiving surface. As the optical low pass filter 4 is retracted from the optical path, a subject image on which noise is not removed by the optical low pass filter 4 is formed on the light receiving surface of the CMOS image sensor 5.

  In the CMOS image sensor 5, a plurality of photoelectric conversion elements are arranged in the row direction and the column direction, respectively. On the light receiving surface of the photoelectric conversion element, color filters that transmit a specific color component (for example, any one of the red component, the green component, and the blue component) are periodically formed in the row direction and the column direction, respectively. ing. When the subject is imaged by the CMOS image sensor 5, the signal charge accumulated in the photoelectric conversion element is converted into a video signal representing the subject image and output from the CMOS image sensor 5. The digital camera 1 includes an actuator 6. The actuator 6 is a device that positions the CMOS image sensor 5 in a direction perpendicular to the optical axis of the CMOS image sensor 5. Details of the positioning position of the CMOS image sensor 5 will be described later.

  The video signal output from the CMOS image sensor 5 is input to the image processing device 13. The image processing device 13 includes an amplification circuit 14 that amplifies the video signal, a black level correction circuit 15 that corrects the black level of the video signal, and an analog / digital converter that generates digital image data by analog / digital conversion of the video signal. A digital conversion circuit (not shown) is also included.

  The digital image data output from the image processing device 13 is given to the driver 9. The subject image is displayed on the display screen of the display device 10.

  When the shutter release button included in the operation device 7 is pressed, the digital image data output from the image processing device 13 is given to the memory 16 and temporarily stored. Digital image data is read from the memory 16 and encoded by the encoder 8. The encoded digital image data is recorded on the memory card 18 by the recording controller 17.

  FIG. 2 shows a part of the light receiving surface of the CMOS image sensor 5.

  In the CMOS image sensor 5, a plurality of photoelectric conversion regions 21 are defined in the row direction and the column direction, respectively. Photoelectric conversion elements are formed in these photoelectric conversion regions 21. On the light receiving surface of each photoelectric conversion region 21 (photoelectric conversion element), a color filter having a characteristic of transmitting a specific color component is periodically repeated in the row direction and the column direction. In the example shown in FIG. 2, any one of the signs “A”, “B”, “C”, and “D” is attached in the photoelectric conversion region 21. These codes “A”, “B”, “C” or “D” are represented by the color component represented by the code “A”, the color component represented by the code “B”, and the code “C”. This shows that a color filter having a characteristic of transmitting the color component or the color component represented by the symbol “D” is formed in the photoelectric conversion region 21. The characteristics of the color components indicated by the signs “A”, “B”, “C”, or “D” may be characteristics that allow the same color components to pass through even if the signs are different. For example, the code “A” and the code “D” or the code “B” and the code “C” may have the characteristics of transmitting the same color component. In the example shown in FIG. 2, a color filter having a characteristic of transmitting a color component represented by reference signs “A” and “B” or reference signs “C” and “D” is periodically repeated in the row direction. In the column direction, a color filter having a characteristic of transmitting the color component represented by the symbols “A” and “C” or “B” and “D” is periodically repeated. . Two photoelectric conversion regions 21 each in the row direction and the column direction (photoelectric conversion regions 21 to which reference numerals “A”, “B”, “C”, and “D” are attached) are included in one block Br1. It can be said that the block Br1 is repeated in the row direction and the column direction.

  3 to 6 show the shift position (positioning position) of the CMOS image sensor 5.

  In FIG. 3 to FIG. 6, for easy understanding, a shift position for one block Br <b> 1 is shown.

  In the digital camera 1 shown in FIG. 1, the CMOS image sensor 5 is arranged in the direction perpendicular to the optical axis of the CMOS image sensor 5 (the row direction and the column direction of the photoelectric conversion elements constituting the CMOS image sensor 5). Although the CMOS image sensor 5 is not necessarily positioned as will be described later, the CMOS image sensor 5 is positioned by being displaced vertically relative to the optical axis of the CMOS image sensor 5. Just do it.

In this embodiment, the repetition interval of the color filter corresponds to the distance between N photoelectric conversion elements (photoelectric conversion regions 21) that are even in the row direction and the column direction, and the repetition interval of the color filter. Is different from the case where it corresponds to the distance of Q photoelectric conversion elements (photoelectric conversion regions 21) which are odd numbers in the row direction and the column direction, respectively. When the repetition interval of the color filter corresponds to the distance between N photoelectric conversion elements that are even numbers in the row direction and the column direction, all m that are natural numbers including 0 of N / 2 or less are used. All the positions obtained by multiplying the coefficient m · N / (N + 1) multiplied by N / (N + 1) by the pitch of the photoelectric conversion elements and all m which are natural numbers including 0 of N / 2 or less are negative − All positions obtained by multiplying the coefficient -m · N / (N + 1) by m multiplied by N / (N + 1) by the pitch of the photoelectric conversion element are the relative positioning positions of the CMOS image sensor 5. In this case, the number of positions is (N + 1) ² . When the repetition interval of the color filter corresponds to the distance between Q photoelectric conversion elements that are odd numbers in the row direction and the column direction, all the ps that are natural numbers including 0 of Q / 2 or less are used. All the positions obtained by multiplying the coefficient p · Q / (Q + 1) multiplied by Q / (Q + 1) by the pitch of the photoelectric conversion elements and all ps which are natural numbers including 0 of Q / 2 or less are negative − Coefficients obtained by multiplying p by Q / (Q + 1) −p · Q / (Q + 1) by the pitch of the photoelectric conversion element, coefficients 2p · Q / (Q + 1) or coefficients −2p · Q / The position obtained by multiplying (Q + 1) by the pitch of the photoelectric conversion element is the relative positioning position of the CMOS image sensor 5. In this case, the number of positions is (Q + 1) ² .

  In the arrangement of the color filters shown in FIG. 2, the repetition interval of the color filters corresponds to the distance between two (even) photoelectric conversion elements (photoelectric conversion regions 21) in the row direction and the column direction. = 2, m = 1. The positioning position of the CMOS image sensor 5 is as follows: the row direction is the X direction, the column direction is the Y direction, and the reference position before shifting the CMOS image sensor 5 is (X, Y) = (0, 0). = −2 / 3, 0, 2/3, Y = −2 / 3, 0, 2/3.

  Referring to FIG. 3, the position P1 of the block Br1 is a reference position where the CMOS image sensor 5 is not shifted, and (X, Y) = (0, 0). The position P2 of the block Br1 is (X, Y) = (− 2/3, 0). The position P3 of the block Br1 is (X, Y) = (2/3, 0).

  Referring to FIG. 4, the position P4 of the block Br1 is (X, Y) = (0, 2/3). The position P5 of the block Br1 is (X, Y) = (0, −2/3).

  Referring to FIG. 5, the position P6 of the block Br1 is (X, Y) = (− 2/3, 2/3). The position P7 of the block Br1 is (X, Y) = (2/3, −2/3).

  Referring to FIG. 6, the position P8 of the block Br1 is (X, Y) = (2/3, 2/3). The position P9 of the block Br1 is (X, Y) = (− 2/3, −2/3).

As shown in FIGS. 3 to 6, the CMOS image sensor 5 is perpendicular to the optical axis of the CMOS image sensor 5 so that the specific block Br1 is at each of the nine positions P1 to P9. To be displaced. Further, since the color filter repetition interval corresponds to the distance between the two photoelectric conversion regions 21 (photoelectric conversion elements), the number of positioning is (2 + 1) ² = 9.

  In this embodiment, imaging is performed by the CMOS image sensor 5 at each positioned position. In the example shown in FIGS. 3 to 6, images are taken at each of the nine positioning positions, and a total of nine times are taken. As described above, it goes without saying that the CMOS image sensor 5 is displaced by the actuator 6 (positioning means).

  FIG. 7 shows a part of a composite subject image obtained when the CMOS image sensor 5 having the color filter array shown in FIG. 2 is positioned at nine positions as shown in FIGS. ing.

  The composite subject image is composed of a large number of pixels 22. Each of these pixels 22 corresponds to each of the photoelectric conversion regions 21 (photoelectric conversion elements) as shown in FIG. In FIG. 7, reference sign “A”, reference sign “B”, reference sign “C”, or reference sign “D” attached to the pixel 22 is a reference sign “A” that represents the transmission characteristics of the color components of the color filter shown in FIG. , “B”, “C”, or “D”. Further, in FIG. 7, the arithmetic numbers “1”, “2”, “3” attached to the pixel 22 after the reference “A”, the reference “B”, the reference “C”, or the reference “D”. ”,“ 4 ”,“ 5 ”,“ 6 ”,“ 7 ”,“ 8 ”and“ 9 ”indicate the position“ P1 ”of the CMOS image sensor 5 as described with reference to FIGS. ”,“ P2 ”,“ P3 ”,“ P4 ”,“ P5 ”,“ P6 ”,“ P7 ”,“ P8 ”, and“ P9 ”. For example, the pixel 22 (A1) to which the symbol “A” is attached and the arithmetic numeral “1” is attached is the photoelectric conversion region 21 (the symbol “A” in the photoelectric conversion region 21). This indicates that the CMOS image sensor 5 represented by the signal charge accumulated in the photoelectric conversion element) is obtained by imaging at a position defined by the position P1. Similarly, for example, the pixel 22 (B2) to which the reference numeral “B” is attached and the arithmetic numeral “2” is attached is the photoelectric conversion of the photoelectric conversion region 21 to which the reference numeral “B” is attached. This indicates that the CMOS image sensor 5 represented by the signal charge accumulated in the region 21 (photoelectric conversion element) is obtained by imaging at a position defined by the position P2. The same applies to the other pixels 22.

  The CMOS image sensor 5 having the color filter array as shown in FIG. 2 is positioned at nine positions as described with reference to FIGS. 3 to 6, and the subject is imaged at each position. , Having three times as many pixels in the row direction (horizontal direction) and column direction (vertical direction) as compared to the subject image obtained when the CMOS image sensor 5 is imaged without shifting (imaging at the reference position) A high-resolution subject image can be obtained. As shown in FIG. 7, the color pixel array of the pixels 22 representing the high-resolution subject image (the array of colors represented by the pixels 22) is the same as the color filter array of the photoelectric conversion region 21 shown in FIG. Digital image data (video signal) obtained when the CMOS image sensor 5 is positioned in the same manner as image processing for digital image data (video signal) obtained when the image is taken without shifting the CMOS image sensor 5. Image processing can be performed.

  8 to 14 show modifications.

  2 to 7 show that the repetition interval of the color filter corresponds to the distance of the even number (two pieces) of photoelectric conversion regions 21 (photoelectric conversion elements) in the row direction and the column direction, respectively. Reference numeral 14 denotes the distance between the odd-numbered (three) photoelectric conversion regions 21 (photoelectric conversion elements) in the row direction and the column direction, respectively.

  FIG. 8 shows a part of the light receiving surface of the CMOS image sensor 5.

  In the example shown in FIG. 8, the symbols “A”, “B”, “C”, “D”, “E”, “F”, “G”, “H”, or “I” exist in the photoelectric conversion region 21. One of these is attached. These symbols “A”, “B”, “C”, “D”, “E”, “F”, “G”, “H”, or “I” are color components represented by the symbol “A”. , A color component represented by the symbol “B”, a color component represented by the symbol “C”, a color component represented by the symbol “D”, a color component represented by the symbol “E”, and a color component represented by the symbol “F”. A color filter having a characteristic of transmitting the color component represented by the reference numeral “G”, the color component represented by the reference numeral “H”, or the color component represented by the reference numeral “I”. 21 shows that it is formed. The characteristics of the color components indicated by the symbols “A”, “B”, “C”, “D”, “E”, “F”, “G”, “H”, or “I” The characteristic which permeate | transmits the same color component may be sufficient. In the example shown in FIG. 8, in the row direction, “A”, “B” and “C”, or “D”, “E” and “F”, or “G”, “ A color filter having a characteristic of transmitting a color component represented by “H” and a symbol “I” is periodically repeated, and in the column direction, a symbol “A”, a symbol “D”, a symbol “G”, Alternatively, a color filter having a characteristic of transmitting the color component represented by the code “B”, the code “E” and the code “H”, or the code “C”, the code “F” and the code “I” is periodically It has been repeated. Three photoelectric conversion regions 21 each in the row direction and the column direction (reference “A”, reference “B”, reference “C”, reference “D”, reference “E”, reference “F”, reference “G”) , The photoelectric conversion region 21 to which the reference symbol “H” and the reference symbol “I” are attached forms one block Br2, and the block Br2 is repeated in the row direction and the column direction.

  9 to 13 show the shift position (positioning position) of the CMOS image sensor 5.

  9 to 13 also show the shift position for one block Br2 for easy understanding.

In the example shown in FIG. 8, the repetition interval of the color filter corresponds to the distance between three odd-numbered photoelectric conversion elements (photoelectric conversion regions 21) in the row direction and the column direction, respectively. If the repetition interval of the color filter corresponds to the distance of Q photoelectric conversion elements that are odd numbers in the row direction and the column direction, respectively, as described above, it is a natural number including 0 that is Q / 2 or less. All the positions obtained by multiplying the coefficient p · Q / (Q + 1) by multiplying a certain p by Q / (Q + 1) by the pitch of the photoelectric conversion element and all ps which are natural numbers including 0 less than Q / 2 -P with Q / (Q + 1) multiplied by -p · Q / (Q + 1) all the positions obtained by multiplying the pitch of the photoelectric conversion element by the factor 2p · Q / (Q + 1) or coefficient The position obtained by multiplying −2p · Q / (Q + 1) by the pitch of the photoelectric conversion element is the relative positioning position of the CMOS image sensor 5. The number of positions is (Q + 1) ² .

  In the arrangement of the color filters shown in FIG. 8, the repetition interval of the color filters corresponds to the distance of three (odd number) photoelectric conversion elements (photoelectric conversion regions 21) in the row direction and the column direction. = 3, p = 1. The positioning position of the CMOS image sensor 5 is as follows: the row direction is the X direction, the column direction is the Y direction, and the reference position before shifting the CMOS image sensor 5 is (X, Y) = (0, 0). = -6/4, -3/4, 0, 3/4, Y = -6/4, -3/4, 0, 3/4 or X = -3/4, 0, 3/4, 6 / 4, Y = −3 / 4, 0, 3/4, 6/4.

  Referring to FIG. 9, the position P11 of the block Br2 is a reference position where the CMOS image sensor 5 is not shifted, and (X, Y) = (0, 0). The position P12 of the block Br2, (X, Y) = (− 6/4, 0). The position P13 of the block Br2 is (X, Y) = (− 3/4, 0). The position P14 of the block Br2 is (X, Y) = (3/4, 0).

  Referring to FIG. 10, the position P15 of the block Br2 is (X, Y) = (0, −6/4). The position P16 of the block Br2 is (X, Y) = (0, −3/4). The position P17 of the block Br2 is (X, Y) = (0, 3/4).

  Referring to FIG. 11, the position P18 of the block Br2 is (X, Y) = (− 6/4, −6/4). The position P19 of the block Br2 is (X, Y) = (− 4/3, −6/4). The position P20 of the block Br2 is (X, Y) = (4/3, −6/4).

  Referring to FIG. 12, the position P21 of the block Br2 is (X, Y) = (− 6/4, −4/3). The position P22 of the block Br2 is (X, Y) = (− 4/3, −4/3). The position P23 of the block Br2 is (X, Y) = (4/3, −4/3).

  Referring to FIG. 13, the position P24 of the block Br2 is (X, Y) = (− 6/4, 4/3). The position P25 of the block Br2 is (X, Y) = (− 4/3, 4/3). The position P26 of the block Br2 is (X, Y) = (4/3, 4/3).

  In the example shown in FIGS. 9 to 13, when the reference position before shifting the CMOS image sensor 5 is (X, Y) = (0, 0), X = −6 / 4, −3/4, The CMOS image sensor 5 is shifted to the positions 0, 3/4, Y = −6 / 4, −3/4, 0, 3/4, but X = −3 / 4, 0, 3/4, The CMOS image sensor 5 may be positioned at 6/4, Y = −3 / 4, 0, 3/4, 6/4.

As shown in FIGS. 9 to 13, the CMOS image sensor 5 is perpendicular to the optical axis of the CMOS image sensor 5 so that the specific block Br2 is at each of the 16 positions P11 to P26. It is positioned in the direction (row direction and column direction of the photoelectric conversion elements constituting the CMOS image sensor 5). Further, since the color filter repetition interval corresponds to the distance between the three photoelectric conversion regions 21 (photoelectric conversion elements), the number of positioning is (3 + 1) ² = 16.

  In this modification, imaging by the CMOS image sensor 5 is performed at each positioned position. In the example shown in FIGS. 9 to 13, images are taken at each of the 16 positions, and a total of 16 times are taken. As described above, it goes without saying that the CMOS image sensor 5 is displaced by the actuator 6 (positioning means).

  FIG. 14 shows a part of a subject image obtained when the CMOS image sensor 5 having the color filter arrangement shown in FIG. 8 is imaged at 16 positions as shown in FIGS. Is shown.

  The subject image shown in FIG. 14 is also composed of a large number of pixels 22. Each of these pixels 22 corresponds to each of the photoelectric conversion regions 21 (photoelectric conversion elements) as shown in FIG. Also in FIG. 14, reference signs “A”, “B”, “C”, “D”, “E”, “F”, “G”, and “H” attached to the pixels 22. Alternatively, the symbol “I” indicates the symbol “A”, the symbol “B”, the symbol “C”, the symbol “D”, the symbol “E”, the symbol “F” that represents the transmission characteristics of the color components of the color filter shown in FIG. ”,“ G ”,“ H ”, or“ I ”. Further, in FIG. 14, the pixel 22 has a symbol “A”, a symbol “B”, a symbol “C”, a symbol “D”, a symbol “E”, a symbol “F”, a symbol “G”, a symbol “H”, or Arithmetic numbers “1”, “2”, “3”, “4”, “5”, “6”, “7”, “8”, “9” attached after the symbol “I” , “10”, “11”, “12”, “13”, “14”, “15” and “16” are the same as those of the CMOS image sensor 5 as described with reference to FIGS. Positions “P11”, “P12”, “P13”, “P14”, “P15”, “P16”, “P17”, “P18” “P19”, “P20” “P21”, “P22”, “P23” , “P24”, “P25” and “P26”. In the same manner as described above, for example, the pixel 22 (A11) to which the reference numeral “A” is attached and the arithmetic numeral “11” is attached is assigned the reference numeral “A” in the photoelectric conversion region 21. This indicates that the CMOS image sensor 5 is obtained by imaging at a position represented by the signal charge accumulated in the photoelectric conversion area 21 (photoelectric conversion element) and defined by the position P11. Similarly, for example, the pixel 22 (B12) to which the symbol “B” is attached and the arithmetic numeral “12” is attached is the photoelectric conversion having the symbol “B” in the photoelectric conversion region 21. This indicates that the CMOS image sensor 5 represented by the signal charge accumulated in the region 21 (photoelectric conversion element) is obtained by imaging at a position defined by the position P12. The same applies to the other pixels 22.

  The CMOS image sensor 5 having the color filter array as shown in FIG. 8 is positioned at 16 positions as described with reference to FIGS. High-resolution subject with four times the number of pixels in the row direction (horizontal direction) and column direction (vertical direction) compared to the subject image obtained when the sensor 5 is imaged without shifting (imaging at the reference position) An image is obtained. As shown in FIG. 14, the color pixel array of the pixels 22 representing the high-resolution subject image (the color array represented by the pixels 22) is the same as the color filter array of the photoelectric conversion region 21 shown in FIG. Similar to the image processing for the digital image data (video signal) obtained when the image is taken without shifting the CMOS image sensor 5, the image for the digital image data (video signal) obtained when the CMOS image sensor 5 is shifted. Processing can be performed.

  FIG. 15 is a flowchart showing the imaging processing procedure of the digital camera 1.

  When the pixel shift super-resolution imaging mode is set by the mode setting switch included in the operating device 7 (mode setting means) (YES in step 31), the device control device 12 (optical low-pass filter control) The optical low pass filter 4 is retracted from the front side of the light receiving surface of the CMOS image sensor 5 (step 32). When the pixel-shifted super-resolution imaging mode (shifted imaging mode) is set, a high-resolution subject image can be obtained, so that the reduction in resolution associated with noise removal by the optical low-pass filter 4 can be prevented. It is.

  The CMOS image sensor 5 is positioned at a reference position (the position of the CMOS image sensor 5 where the optical axis of the lens device 2 and the optical axis of the CMOS image sensor 5 coincide with each other, such as the reference positions P1, P11 described above). Thus, the subject is imaged by the CMOS image sensor 5 under the control of the CPU 11 (imaging control means) (step 33). The video signal output from the CMOS image sensor 5 by imaging is converted into digital image data and temporarily stored in the memory 16. Subsequently, the CMOS image sensor 5 is shifted (positioned) to the shifted position by the actuator 6 (step 34). If imaging of the subject at all the shifted positions (positioning positions) has not been completed (NO in step 35), the subject is imaged again at the shifted position (step 33). The video signal obtained by imaging is converted into digital image data and temporarily stored in the memory 16. The CMOS image sensor 5 is shifted to the next shifting position by the actuator 6 (step 34).

  The CMOS image sensor 5 is shifted to the shifted position until imaging at all the shifted positions of the CMOS image sensor 5 determined according to the repetition interval of the color filter formed in the CMOS image sensor 5 is completed. And imaging at the shifted position is repeated. When the repetition interval of the color filter corresponds to two photoelectric conversion elements constituting the CMOS image sensor 5, as described with reference to FIGS. 2 to 7, a position other than the reference position P1 is used. Imaging is performed by the CMOS image sensor 5 at nine positions from P2 to P9. When the color filter repetition interval corresponds to three photoelectric conversion elements constituting the CMOS image sensor 5, as described with reference to FIGS. 8 to 14, in addition to the reference position P11, In addition, imaging by the CMOS image sensor is performed at 16 positions from positions P12 to P26.

  When shifting to all the shifted positions of the CMOS image sensor 5 and imaging at the shifted positions are completed (YES in step 35), the memory 16 stores a plurality of digital images obtained by imaging at all shifted positions. Image data (video signal) is stored. As described with reference to FIGS. 7 and 14, the composite image data representing one composite subject image whose pixel array corresponds to the color filter array of the CMOS image sensor 5 from the digital image data for a plurality of sheets. (Synthetic subject image signal) is generated by the CPU 11 (synthetic subject image signal generation means). The generated composite image data is recorded on the memory card 18 by the recording control device 17. A composite subject image represented by the composite image data may be displayed on the display screen of the display device 10.

  When the normal imaging mode is set by the mode setting switch included in the operation device 7 (mode setting means) (NO in step 31), the device control device 12 (optical low pass filter control means) The optical low pass filter 4 is maintained in a state of being disposed in front of the CMOS image sensor 5 without being retracted from the front of the CMOS image sensor 5. The CMOS image sensor 5 is positioned at the reference position and the subject is imaged (step 36). When the normal imaging mode is set, the relative positioning of the CMOS image sensor 5 is stopped by the device control device 12 (positioning stop means).

  FIG. 16 shows an opening 23 of the photoelectric conversion region 21 included in the CMOS image sensor 5. This corresponds to the block Br1 shown in FIG.

  The openings 23 shown in FIG. 16 are circular, and the diameter of the openings 23 is equal to the pitch of the photoelectric conversion regions 21 (photoelectric conversion elements).

  FIG. 17 shows another embodiment, and shows an opening 24 of the photoelectric conversion region 21 included in the CMOS image sensor 5. This corresponds to the block Br1 shown in FIG.

  The opening 24 shown in FIG. 17 is also circular like the opening 23 shown in FIG. The diameter of the opening 24 shown in FIG. 17 is equal to 1/3 of the pitch of the photoelectric conversion region 21 (photoelectric conversion element).

  As shown in FIG. 2, when the repetition interval of the color filter corresponds to the distance of N photoelectric conversion elements that are even numbers in the row direction and the column direction, the photoelectric conversion region 21 (photoelectric conversion element 21). The opening diameter of the light receiving surface (the opening diameter means the diameter of the opening) is preferably equal to or larger than the value obtained by multiplying the pitch of the photoelectric conversion region 21 (photoelectric conversion element) by 1 / (N + 1). By using such an opening diameter, generation of false color is prevented in advance. In the example shown in FIG. 17, since N = 2, the diameter of the opening 24 is set to 1/3 of the pitch of the photoelectric conversion region 21 (photoelectric conversion element). The diameter of the openings 24 may be 1/3 or more and 1 or less of the pitch of the photoelectric conversion regions 21 (photoelectric conversion elements).

  18 shows the positional relationship of the opening 24 when the image sensor 5 in which the opening 24 shown in FIG. 17 is formed is shifted (positioned) as shown in FIGS. .

  When the diameter of the opening 24 is 1/3 of the pitch of the photoelectric conversion region 21 (photoelectric conversion element), there is no gap between the openings 24 when the image sensor 5 is shifted (positioned). . Thereby, even if the image sensor 5 is displaced (positioned), the generation of false colors is prevented.

  FIG. 19 shows another embodiment, and shows an opening 27 of the photoelectric conversion region 21 included in the CMOS image sensor 5. This corresponds to the block Br2 shown in FIG.

  The opening 27 shown in FIG. 19 is also circular like the opening 23 shown in FIG. 16 and the opening 24 shown in FIG. However, the diameter of the opening 27 shown in FIG. 19 is equal to 1/4 of the pitch of the photoelectric conversion region 21 (photoelectric conversion element).

  As shown in FIG. 8, even when the repetition interval of the color filter corresponds to the distance of Q photoelectric conversion elements that are odd numbers in the row direction and the column direction, the photoelectric conversion region 21 (photoelectric conversion element) The aperture diameter of the light receiving surface (the aperture diameter refers to the diameter of the aperture) is preferably equal to or greater than the value obtained by multiplying the pitch of the photoelectric conversion region 21 (photoelectric conversion element) by 1 / (Q + 1). By using such an opening diameter, generation of false color is prevented in advance. In the example shown in FIG. 19, since Q = 3, the diameter of the opening 27 is ¼ of the pitch of the photoelectric conversion region 21 (photoelectric conversion element). The diameter of the opening 27 may be ¼ or more and 1 or less of the pitch of the photoelectric conversion region 21 (photoelectric conversion element).

  FIG. 20 shows the positional relationship of the openings 27 when the image sensor 5 in which the openings 27 shown in FIG. 19 are formed is shifted (positioned) as shown in FIGS. .

  When the diameter of the opening 27 is 1/4 of the pitch of the photoelectric conversion region 21 (photoelectric conversion element), there is no gap between the openings 27 when the image sensor 5 is shifted (positioned). . Thereby, even if the image sensor 5 is displaced (positioned), the generation of false colors is prevented.

  FIG. 21 shows digital image data representing each of nine subject images obtained by shifting the CMOS image sensor 5 as shown in FIGS. 3 to 6 and imaging the subject at each shifted position. Figure 16 shows how it is memorized. In this embodiment, when the digital image data representing each of the nine subject images is stored in the memory 16, composite subject image data (synthesized subject image signal) representing the composite subject image is generated. FIG. 21 shows how the digital image data representing each of the nine subject images is stored in the memory 16, but not the digital image data representing each of the nine subject images. As shown in FIG. 13, digital image data representing each of 16 subject images obtained by relatively shifting the CMPS image sensor 5 at 16 positions is stored in the memory 16 in the same manner, and a composite subject image is represented. Composite subject image data may be generated. Further, digital image data representing each of the subject images other than 9 or 16 may be stored in the memory 16 to generate composite subject image data representing the composite subject image.

  In FIG. 21, as in FIG. 7, the reference numerals “A” to “D” are changed from the reference numerals “A” to “D” attached to the characteristics of transmitting the color components of the color filter shown in FIG. Correspondingly, the arithmetic numbers “1” to “9” indicate image data obtained by imaging at the shift positions “P1” to “P9” shown in FIGS. 3 to 6.

  In the first imaging, the CMOS image sensor 5 is positioned at the reference position P1. Digital image data (represented by reference numerals A1, B1, C1, and D1) obtained by the first imaging is the first subject image among the pixels 22 constituting the composite subject image shown in FIG. Are stored in a memory area corresponding to the combined position.

  In the second imaging, the CMOS image sensor 5 is positioned at a position P2 shown in FIG. The digital image data (represented by the symbols B2 and D2) obtained by the second imaging is located at the synthesis position of the second subject image among the pixels 22 constituting the synthesized subject image shown in FIG. It is stored in the corresponding memory area.

  Also in the third to ninth imaging, the CMOS image sensor 5 is positioned at each of the positions P3 to P9 shown in FIGS. The digital image data of the digital image data obtained by the ninth imaging from the digital image data obtained by the third imaging is three of the pixels 22 constituting the composite subject image shown in FIG. The images are sequentially stored in the memory areas corresponding to the synthesis positions of the subject images from the 9th to the 9th.

  When the respective digital image data representing nine subject images obtained by nine imaging operations are stored in the memory 16, they are stored in the memory 16 in an array corresponding to the pixel array of the composite subject image. By simply reading from the memory 16, the composite subject image can be displayed. In this way, nine image signals (plurality) obtained by imaging using the CMOS image sensor 5 for each relatively displaced position (relatively positioned position) are combined subject images. Can be stored in the memory 16 (combined subject image signal generating means).

  FIG. 22 is a flowchart showing the processing procedure of the digital camera 1, showing another embodiment.

  Pre-imaging is performed by the CMOS image sensor 5 and a video signal is output from the CMOS image sensor 5. The video signal is input to the CPU 11 where the photometric calculation is performed. By the photometric calculation, the exposure time of the CMOS image sensor 5 and the amplification factor by the amplifier circuit 14 are determined by the CPU 11. The determined exposure time and amplification factor are fixed so as to be the same regardless of the position where the CMOS image sensor 5 is positioned as described above (step 43).

  The CMOS image sensor 5 is positioned at the reference position, and the subject is imaged at the reference position with the exposure time fixed by using the CMOS image sensor 5 (step 44). Subsequently, the CMOS image sensor 5 is shifted to a shift position (positioning position) (step 45).

  When the subject is imaged by the CMOS image sensor 5, the video signal output from the CMOS image sensor 5 is fixed regardless of the position of the CMOS image sensor 5 in the amplification circuit 14 included in the image processing device 13. (Step 46). Since the video signal output from the CMOS image sensor 5 is amplified at a fixed amplification factor regardless of the position of the CMOS image sensor 5 and the exposure times of the plurality of subject images are the same, Even if a super-resolution composite subject image is generated, the brightness remains constant and a natural composite subject image can be obtained.

  Until imaging by the CMOS image sensor 5 at all the shifted positions is completed (YES in step 47), imaging using the CMOS image sensor 5 is performed at the same shifted exposure time at each shifted position. (Step 44). When imaging at all the shifted positions is completed (YES in step 47), a super-resolution composite subject image is generated in the image processing device 13 using a plurality of video signals obtained according to the number of imaging.

  FIG. 23 is a flowchart showing the processing procedure of the digital camera 1, showing another embodiment. In FIG. 23, the same processes as those shown in FIG.

  Photometric calculation is performed in the same manner as described above (step 41), and when the CMOS image sensor 5 is at the reference position, the exposure time for imaging is determined by the CPU 11 based on the calculation result (step 42A). ). Subsequently, when the CMOS image sensor 5 is shifted to a position other than the reference position and picks up an image, the exposure time for image pickup at each position is determined by the CPU 11 (step 43A). The exposure time when the CMOS image sensor 5 is at the reference position and the exposure time when the CMOS image sensor 5 is positioned at a position other than the reference position may be the same (in this case, the processing shown in FIG. The exposure time when the CMOS image sensor 5 is at the reference position (preferably the exposure time at which the brightness is appropriate) is positioned at a position other than the reference position. It is assumed that the exposure time is longer than the exposure time. Of course, the exposure time when the CMOS image sensor 5 is at the reference position may be shorter than the exposure time when the CMOS image sensor 5 is positioned at a position other than the reference position.

  The CMOS image sensor 5 at the reference position is exposed for the determined exposure time, thereby imaging the subject (step 44). When the imaging of the subject is completed, the CMOS image sensor 5 is shifted (positioned) (step 45).

  When the subject is imaged, a video signal representing the subject image is output from the CMOS image sensor 5. An amplification circuit so that both the level of the video signal obtained when the CMOS image sensor 5 is at the reference position and the level of the video signal obtained when the CMOS image sensor 5 is positioned are equal. In step 14, the amplification factor is determined by the CPU 11 in accordance with the exposure time when the CMOS image sensor 5 captures an image. The CPU 11 (amplification control means) controls the amplification of the amplifying circuit 14 so that the video signal obtained by imaging with a relatively short exposure time is controlled so that the level of the video signal obtained by imaging with a relatively long exposure time is obtained. Is done. The video signal is amplified by the amplification circuit 14 at the determined amplification factor (step 46A). When the CMOS image sensor 5 is positioned and imaged, the level of the video signal after amplification in the amplifier circuit 14 is constant even if the exposure times are not the same. Even when a super-resolution composite subject image is generated from a plurality of subject images, the brightness is constant and a natural composite subject image can be obtained.

  If imaging at all the shifted positions has not been completed (NO in step 47), the subject is imaged with the exposure time at the position where the CMOS image sensor 5 is shifted (step 44). Such imaging of the subject (step 44), shifting of the CMOS image sensor (step 45), and amplification with an amplification factor corresponding to the exposure time (step 46A) are repeated until imaging at all the shifted positions is completed. (Step 47).

  When imaging at all the shifted positions (positioning positions) is completed (YES in step 47), a plurality of amplified subjects obtained by imaging using the CMOS image sensor 5 at each of the reference position and the shifted position A composite subject image signal representing a super-resolution composite subject image is generated by the image processing device 13 using a video signal representing the image.

  FIG. 24 shows another embodiment, and is a block diagram showing an electrical configuration of the digital camera 1A. In FIG. 23, the same components as those shown in FIG.

  In the digital camera 1 shown in FIG. 1, the CMOS image sensor 5 is shifted in the vertical direction from the optical axis of the CMOS image sensor 5 by the actuator 6, but in the digital camera 1A shown in FIG. The lens device 2 (imaging lens) is displaced (positioned) in the vertical direction from the optical axis of the lens device 2 by the actuator 6A. Both the lens device 2 and the CMOS image sensor 5 may be shifted (positioned) in the vertical direction from the optical axis of the lens device 2 (CMOS image sensor 5).

  FIG. 25 shows another embodiment and is a block diagram showing an electrical configuration of the digital camera 1B. In FIG. 25, the same components as those shown in FIG.

  The entire digital camera 1B shown in FIG. 25 is placed on a pan head 19 (holding base). The pan head 19 can shift the entire digital camera 1B in the direction perpendicular to the optical axis of the digital camera 1B (lens device 2, CMOS image sensor 5) by an actuator 6C. The actuator 6 </ b> C controls the pan head 19 under the control of the device control device 12.

  As shown in FIG. 1, the CMOS image sensor 5 may be shifted (positioned), or the lens device 2 may be shifted (positioned) as shown in FIG. As shown in FIG. 25, the entire digital camera 1B may be shifted (positioned). The CMOS image sensor 5 may be shifted to the relatively shifted position (it should be positioned at the positioning position). If the CMOS image sensor 5 is shifted by at least one shift (positioning) among the shift (positioning) of the CMOS image sensor 5, the shift (positioning) of the lens device 2, and the shift (positioning) of the digital camera 1B. Good (just need to be positioned).

  FIG. 26 shows a part of the CMOS image sensor 5.

  The color filters formed in the CMOS image sensor 5 shown in FIG. 26 are of the Bayer array. In the Bayer array, a color filter having a characteristic of transmitting a green light component (indicated by a symbol “G”) and a color filter having a characteristic of transmitting a red light component in odd rows (may be even rows) ( And a color filter (shown by a symbol “B”) having a characteristic of transmitting a blue light component and a green filter in an even row (or an odd row). Color filters (indicated by reference numeral “G”) having characteristics of transmitting light components are alternately formed.

  When a Bayer array color filter is formed in the CMOS image sensor 5, the color filter repeat interval corresponds to two photoelectric conversion regions 21 (photoelectric conversion elements) as shown in FIG. Therefore, as shown in FIGS. 3 to 6, the position of the CMOS image sensor 5 is shifted (positioned) so as to be relatively positioned from P1 to P9. The subject is imaged at (position).

  FIG. 27 is a flowchart showing the processing procedure of the digital camera 1, showing another embodiment. The same processing may be performed for the digital camera 1A shown in FIG. 24 or the digital camera 1B shown in FIG.

  When the pixel shift super-resolution imaging mode is set by the imaging mode setting switch included in the operation device 7 (mode setting means) (YES in step 51), the mechanical shutter 3 is opened by the device control device 12 (step 51). 52). The mechanical shutter 3 is disposed in front of the CMOS image sensor 5 and performs an operation of exposing the CMOS image sensor 5 for an exposure time corresponding to the shutter speed. The optical low pass filter 4 may also be retracted from the front of the CMOS image sensor 5. The electronic shutter mode is set by the CPU 11 (step 53), and the shutter operation by the mechanical shutter 3 is stopped.

  The CMOS image sensor 5 is positioned at the reference position, and the subject is imaged using the CMOS image sensor 5 by the electronic shutter at the reference position (step 53). The electronic shutter adjusts the accumulation time (exposure time) of signal charges in the photoelectric conversion element included in the solid-state electronic image pickup device by controlling the solid-state electronic image pickup device. The device controller 12 (electronic shutter means) outputs the signal charge accumulated during the exposure time according to the shutter speed as a video signal from the CMOS image sensor 5. When the imaging of the subject is completed, the CMOS image sensor 5 is shifted (positioned) (step 54). The imaging of the subject using the electronic shutter and the shifting (positioning) to the shifting position (positioning position) of the CMOS image sensor 5 are repeated until the imaging at all the shifting positions is completed (step 55).

  When the normal imaging mode is set by the imaging mode setting switch included in the operating device 7 (mode setting means) (NO in step 51), the mechanical shutter mode is set by the CPU 11 (step 56). The mechanical shutter 3 is controlled by the device controller 12, and the subject is imaged by the CMOS image sensor 5 (step 57).

  Thus, when the pixel shift super-resolution mode (shifted imaging mode) is set by the CPU 11 (shutter control means), the electronic shutter is performed, and when the normal imaging mode is set, the operation by the mechanical shutter 3 is performed. Is called.

  In the pixel shift super-resolution imaging mode, the CMOS image sensor 5 is shifted (positioned), so it is preferable that no vibration is applied to the digital camera 1. When the pixel-shifted super-resolution imaging mode is set, vibration is suppressed because the electronic shutter is used instead of the mechanical shutter 3. However, the mechanical shutter 3 may be used instead of the electronic shutter for imaging at the last shift position (positioning position) among all the shift positions (positioning positions).

  FIG. 28 is a flowchart showing the processing procedure of the digital camera 1, showing still another embodiment. The same processing may be performed for the digital camera 1A shown in FIG. 24 or the digital camera 1B shown in FIG.

  The processing procedure shown in FIG. 28 is obtained by adding a processing procedure to the processing procedure shown in FIG. For this reason, in the process shown in FIG. 28, the same processes as those shown in FIG.

  When the processing from step 31 to step 35 in FIG. 15 or the processing from step 31 and step 36 is completed, the mechanical shutter 3 is closed by the device control device 12 (step 37). With the mechanical shutter 3 closed, the CMOS image sensor 5 performs light-shielded imaging under the control of the device controller 12 (step 38). Black-level digital image data (reference video signal) is obtained by performing shading imaging. Using the obtained digital image data, in the black level correction circuit 15 (black level correction means), a plurality of images obtained by imaging at each shift position (positioning position) in the pixel shift super-resolution imaging mode. The black level correction of each of the digital image data or the black level correction of the subject image obtained in the normal imaging mode is performed (step 39). When the pixel shift super-resolution imaging mode is set, the black level correction by the black level correction circuit 15 (black level correction means) may be performed on the composite subject image data representing the composite subject image.

  FIG. 29 is a flowchart showing a processing procedure of the digital camera 1, showing still another embodiment. The same processing may be performed for the digital camera 1A shown in FIG. 24 or the digital camera 1B shown in FIG.

  When the pixel shift super-resolution imaging mode is set by the imaging mode setting switch included in the operating device 7 (mode setting means) (YES in step 61), the camera shake correction mode is set by the operating device (camera shake correction mode setting means) 7. It is confirmed whether it has been set (step 62). When the camera shake correction mode is set (YES in step 62), the optical low pass filter 4 is retracted from the front of the CMOS image sensor 5 by the device controller 12 (step 63).

  Camera shake correction is performed by the device controller 12 (camera shake correcting means) so that the CMOS image sensor 5 is positioned at the reference position, and the lens device 2 is moved in the direction perpendicular to the central axis of the CMOS image sensor 5 at the reference position. Is done. The subject is imaged using the CMOS image sensor 5 with the camera shake corrected (step 64). When the imaging of the subject is completed, the CMOS image sensor 5 is shifted (positioned) (step 65). The imaging of the subject while correcting the camera shake and the shifting (positioning) to the shifting position (positioning position) of the CMOS image sensor 5 are repeated until the imaging at all the shifting positions (positioning) is completed (step 66). .

  Even if the pixel shift super-resolution imaging mode is set and the camera shake correction mode is not set (NO in step 62), the optical low-pass filter 4 is retracted from the front of the CMOS image sensor 5. However, camera shake correction is not performed. In a state where no camera shake correction is performed, the CMOS image sensor 5 is positioned at the reference position, and the subject is imaged using the CMOS image sensor 5 at the reference position (step 68). When the imaging of the subject is completed, the CMOS image sensor 5 is shifted (positioned) (step 69). The imaging of the subject and the shifting (positioning) to the shifting position (positioning position) of the CMOS image sensor 5 are repeated until the imaging at all the shifting positions is completed (step 70).

  When the normal imaging mode is set by the imaging mode setting switch included in the operation device 7 (mode setting means) (NO in step 61), the CMOS image sensor 5 is not shifted from the reference position. Thus, the subject is imaged (step 71).

  30 and 31 show an opening 81 and an opening 82 formed in the photoelectric conversion region 21 of the CMOS image sensor 5.

  FIG. 30 corresponds to FIGS. 16 and 17 and shows an opening 81 formed in the photoelectric conversion region 21 included in the block Br1.

  As will be described in detail later, the repetition interval of the color filters formed in the CMOS image sensor 5 is the distance between N photoelectric conversion regions 21 (photoelectric conversion elements) that are even numbers in the row direction and the column direction, respectively. When it corresponds, the aperture diameter of the light receiving surface of the photoelectric conversion area 21 (photoelectric conversion element) is a value obtained by multiplying the pitch of the photoelectric conversion area 21 (photoelectric conversion element) by a value of 2 / (N + 1) or less. It is preferable. More preferably, a value obtained by multiplying the pitch of the photoelectric conversion region 21 (photoelectric conversion element) by 2 / (N + 1) is set as the opening diameter of the light receiving surface of the photoelectric conversion region 21 (photoelectric conversion element).

  In FIG. 30, in the block Br1, the color filter repetition interval formed in the CMOS image sensor 5 corresponds to the distance between two photoelectric conversion regions 21 (photoelectric conversion elements) (N = 2). . For this reason, the diameter (opening diameter) of the opening 81 on the light receiving surface of the photoelectric conversion region 21 (photoelectric conversion element) is set to a value obtained by multiplying the pitch of the photoelectric conversion region 21 by 2/3. The diameter (opening diameter) of the opening 81 may be a value obtained by multiplying the pitch of the photoelectric conversion region 21 by a value of 2/3 or less.

  FIG. 31 corresponds to FIG. 19 and shows an opening 82 formed in the photoelectric conversion region 21 included in the block Br2.

  As will be described later in detail, the repetition interval of the color filters formed in the CMOS image sensor 5 is the distance between Q photoelectric conversion regions 21 (photoelectric conversion elements) that are odd numbers in the row direction and the column direction, respectively. When it corresponds, the aperture diameter of the light receiving surface of the photoelectric conversion area 21 (photoelectric conversion element) is a value obtained by multiplying the pitch of the photoelectric conversion area 21 (photoelectric conversion element) by a value of 2 / (Q + 1) or less. It is preferable. More preferably, a value obtained by multiplying the pitch of the photoelectric conversion region 21 (photoelectric conversion element) by 2 / (Q + 1) is set as the opening diameter of the light receiving surface of the photoelectric conversion region 21 (photoelectric conversion element).

  In FIG. 31, in the block Br1, the color filter repetition interval formed in the CMOS image sensor 5 corresponds to the distance between the two photoelectric conversion regions 21 (photoelectric conversion elements) (Q = 3). . For this reason, the diameter (opening diameter) of the opening 81 on the light receiving surface of the photoelectric conversion region 21 (photoelectric conversion element) is set to a value obtained by multiplying the pitch of the photoelectric conversion region 21 by 2/4 = 1/2. The diameter (opening diameter) of the opening 81 may be a value obtained by multiplying the pitch of the photoelectric conversion region 21 by a value of 3/4 = 1/2 or less.

  FIG. 32 shows the relationship between the normalized spatial frequency of the subject and the response (response indicating the resolution characteristics for the subject having various spatial frequencies). The horizontal axis is the spatial frequency, and the vertical axis is the response.

  A graph g0 shows the Nyquist frequency in imaging in the normal imaging mode (a mode in which imaging is performed once at the reference position without relatively shifting the CMOS image sensor 5). In the normal imaging mode, imaging is performed once at the reference position without relatively shifting the CMOS image sensor 5, so that 0.5, which is half the pitch of the photoelectric conversion region 21, is a standardized Nyquist frequency. .

  The graph g1 shows the Nyquist frequency when imaging is performed by relatively shifting the CMOS image sensor 5 at nine positions as shown in FIGS. 3 to 6 in the pixel-shifted super-resolution imaging mode. As shown in FIG. 3 to FIG. 6, images are taken with the CMOS image sensor 5 relatively displaced (relatively positioned) at three positions in the row direction (horizontal direction) and the column direction (vertical direction). Therefore, 1.5 times three times the Nyquist frequency in the case of one imaging at the reference position without relatively shifting the CMOS image sensor 5 is 9 positions as shown in FIGS. The Nyquist frequency is standardized when the CMOS image sensor 5 is imaged with a relative shift.

  The graph g2 shows the Nyquist frequency when imaging is performed with the CMOS image sensor 5 relatively displaced (relatively positioned) at 16 positions as shown in FIGS. 9 to 13 in the pixel-shifted super-resolution imaging mode. Is shown. As shown in FIGS. 9 to 13, images are taken with the CMOS image sensor 5 relatively displaced (positioned) at four positions respectively in the row direction (horizontal direction) and the column direction (vertical direction). Thus, 2.0, which is four times the Nyquist frequency in the case of one imaging at the reference position without relatively shifting the CMOS image sensor 5, is displayed at 16 positions as shown in FIGS. A standardized Nyquist frequency is used when imaging is performed with the sensor 5 relatively shifted.

  The graph g10 shows the relationship between the spatial frequency and the response when the diameter of the opening 23 is equal to the pitch of the photoelectric conversion region 21 (photoelectric conversion element) as shown in FIG. When the diameter of the opening 23 is equal to the pitch of the photoelectric conversion region 21, the response is 0 when the normalized frequency is 1.

  A graph g11 shows the relationship between the exchange frequency and the response when the diameter of the opening 81 corresponds to 2/3 of the pitch of the photoelectric conversion region 21 (photoelectric conversion element) as shown in FIG. When the diameter of the opening 81 corresponds to 2/3 of the pitch of the photoelectric conversion region 21, the response is 0 when the normalized frequency is 1.5.

  The graph g12 shows the relationship between the exchange frequency and the response when the diameter of the opening 82 corresponds to 1/2 of the pitch of the photoelectric conversion region 21 (photoelectric conversion element) as shown in FIG. When the diameter of the opening 82 corresponds to ½ of the pitch of the photoelectric conversion region 21, the response becomes 0 when the normalized frequency is 2.0.

  The graph g13 shows the relationship between the exchange frequency and the response when the diameter of the opening 24 corresponds to 1/3 of the pitch of the photoelectric conversion region 21 (photoelectric conversion element) as shown in FIG. When the diameter of the opening 24 corresponds to 1/3 of the pitch of the photoelectric conversion region 21, the response becomes 0 when the standardized frequency is 3.0.

  Since the portion of the subject corresponding to the frequency exceeding the Nyquist frequency cannot be resolved, the CMOS image sensor 5 is relatively shifted to nine positions as shown in FIGS. 3 to 6 in the pixel-shifted super-resolution imaging mode. In the case of imaging (with relative positioning), referring to the graphs g1 and g11, it is preferable that the apertures 81 have a 2/3 pitch of the photoelectric conversion region 21 as shown in FIG. To generalize this, the color filter repetition interval corresponds to the distance (pitch) of N (for example, N = 2) photoelectric conversion regions 21 (photoelectric conversion regions) that are even in the row direction and the column direction, respectively. In this case, it can be said that the size of the opening of the photoelectric conversion region 21 (photoelectric conversion region) is preferably equal to 2 / (N + 1).

  Similarly, in the pixel-shifted super-resolution imaging mode, as shown in FIGS. 9 to 13, when the CMOS image sensor 5 is relatively shifted (relatively positioned) at 16 positions, Referring to the graph g2 and the graph g12, it is preferable to set the openings 81 to 1/2 = 2/4 of the pitch of the photoelectric conversion region 21 as shown in FIG. To generalize this, the color filter repeat interval corresponds to the distance (pitch) of Q photoelectric conversion regions 21 (photoelectric conversion regions) that are odd numbers in the row direction and column direction, respectively (for example, Q = 3). In this case, it can be said that the size of the opening of the photoelectric conversion region 21 (photoelectric conversion region) is preferably equal to 2 / (Q + 1).

  From the above, the repetition interval of the color filter corresponds to the distance (pitch) of N (for example, N = 2) photoelectric conversion regions 21 (photoelectric conversion regions) each having an even number in the row direction and the column direction. In this case, the size of the opening of the photoelectric conversion region 21 (photoelectric conversion element) is 1 / (N + 1) to the distance (pitch) of the photoelectric conversion region 21 (photoelectric conversion element) in consideration of suppressing false color. In consideration of resolution, the size of the opening of the photoelectric conversion region 21 (photoelectric conversion device) is obtained by multiplying the distance (pitch) of the photoelectric conversion region 21 (photoelectric conversion device) by 2 / (N + 1). Is preferably equal to In summary, when the color filter repetition interval corresponds to the distance (pitch) of N (for example, N = 2) photoelectric conversion regions 21 (photoelectric conversion regions) that are even in the row direction and the column direction, respectively. In this case, the size of the opening of the photoelectric conversion region 21 (photoelectric conversion element) is preferably equal to a value obtained by multiplying the distance (pitch) of the photoelectric conversion region 21 (photoelectric conversion element) by 2 / (N + 1). I understand. However, in the case where the repetition interval of the color filter corresponds to the distance (pitch) of N (for example, N = 2) photoelectric conversion regions 21 (photoelectric conversion regions) that are even numbers in the row direction and the column direction, respectively. The size of the opening of the photoelectric conversion region 21 (photoelectric conversion element) is equal to or larger than a value obtained by multiplying the distance (pitch) of the photoelectric conversion region 21 (photoelectric conversion element) by 1 / (N + 1), and the photoelectric conversion region 21 ( Needless to say, it may be equal to or less than the value obtained by multiplying the distance (pitch) of the photoelectric conversion element by 2 / (N + 1).

  Similarly, when the color filter repetition interval corresponds to the distance (pitch) of Q (for example, Q = 3) photoelectric conversion regions 21 (photoelectric conversion regions) that are odd numbers in the row direction and the column direction, respectively. In this case, the size of the opening of the photoelectric conversion region 21 (photoelectric conversion element) is 1 / (Q + 1) or more to the distance (pitch) of the photoelectric conversion region 21 (photoelectric conversion element) in consideration of suppressing false color. It is preferable that the value is multiplied by the value, and considering the resolution, the size of the opening of the photoelectric conversion region 21 (photoelectric conversion element) is 2 / (the distance (pitch) of the photoelectric conversion region 21 (photoelectric conversion element). It is preferably equal to a value multiplied by Q + 1). In summary, when the color filter repetition interval corresponds to the distance (pitch) of Q (for example, Q = 2) photoelectric conversion regions 21 (photoelectric conversion regions) that are even numbers in the row direction and the column direction, respectively. In this case, the size of the opening of the photoelectric conversion region 21 (photoelectric conversion element) is preferably equal to a value obtained by multiplying the distance (pitch) of the photoelectric conversion region 21 (photoelectric conversion element) by 2 / (Q + 1). I understand. However, when the color filter repetition interval corresponds to the distance (pitch) of Q photoelectric conversion regions 21 (photoelectric conversion regions) that are odd numbers in the row direction and the column direction, respectively (for example, Q = 3). The size of the opening of the photoelectric conversion area 21 (photoelectric conversion element) is equal to or greater than the value obtained by multiplying the distance (pitch) of the photoelectric conversion area 21 (photoelectric conversion element) by 1 / (Q + 1), and the photoelectric conversion area 21 ( Needless to say, it may be equal to or less than a value obtained by multiplying the distance (pitch) of the photoelectric conversion element by 2 / (Q + 1).

  DESCRIPTION OF SYMBOLS 1 Digital camera, 1A digital camera, 1B digital camera, 2 Lens apparatus, 3 Mechanical shutter, 4 Optical low pass filter 5 CMOS image sensor, 6 Actuator, 6A actuator, 6C actuator, 7 Control device, 8 Encoder, 9 Driver, 10 Display device, 11 CPU, 12 Device control device, 13 Image processing device, 14 Amplification circuit, 15 Black level correction circuit, 16 Memory, 17 Recording control device, 18 Memory card, 19 Head, 21 photoelectric conversion area, 22 pixels, 23 apertures, 24 apertures, 27 apertures, 81 apertures, 82 apertures, Br1 block, Br2 block, P1 reference position, P2-P9 shift position (positioning position), P11 reference position, P12-P26 Shift position (positioning position), g0 graph, g1 graph, g2 group Off, g10-g13 graph

Claims (23)

A plurality of photoelectric conversion elements are arranged in each of the row direction and the column direction, and a color filter having a characteristic of transmitting a specific color component is periodically arranged in the row direction and the column direction on the light receiving surface of the photoelectric conversion element. A solid-state electronic image pickup device that is formed repeatedly and converts a signal charge accumulated in the photoelectric conversion device into a video signal representing a subject image and outputs the image by imaging the subject,
The solid-state electronic image sensor is positioned relatively shifted in the row direction and the column direction, and the repetition interval of the color filter corresponds to the distance of the N photoelectric conversion elements that are even numbers in the row direction and the column direction, respectively. In this case, all m obtained by multiplying the coefficient m · N / (N + 1) by multiplying all m, which are natural numbers including 0 of N / 2 or less, by N / (N + 1), by the pitch of the photoelectric conversion element. And a coefficient obtained by multiplying all m, which are natural numbers including 0 and N / 2 or less, by N / (N + 1), minus -m · N / (N + 1) by the pitch of the photoelectric conversion element Positioning means for relatively positioning the solid-state electronic image pickup device at all the positions obtained in this manner, and imaging control means for causing the solid-state electronic image pickup device to image at each position relatively positioned by the positioning means,
An imaging apparatus comprising: A plurality of photoelectric conversion elements are arranged in each of the row direction and the column direction, and a color filter having a characteristic of transmitting a specific color component is periodically arranged in the row direction and the column direction on the light receiving surface of the photoelectric conversion element. A solid-state electronic image pickup device that is formed repeatedly and converts a signal charge accumulated in the photoelectric conversion device into a video signal representing a subject image and outputs the image by imaging the subject,
The solid-state electronic image sensor is positioned relatively shifted in the row direction and the column direction, and the repetition interval of the color filter corresponds to the distance of the Q photoelectric conversion elements that are odd numbers in the row direction and the column direction, respectively. In this case, the coefficient p · Q / (Q + 1) obtained by multiplying all p, which are natural numbers including 0 of Q / 2 or less, by Q / (Q + 1) is multiplied by the pitch of the photoelectric conversion element. The pitch of the photoelectric conversion element is set to a coefficient −p · Q / (Q + 1) obtained by multiplying all positions and all ps, which are natural numbers including 0 of Q / 2 or less, by negative −p / Q / (Q + 1) −p · Q / (Q + 1). The solid-state electronic imaging device, and all the positions obtained by multiplying the coefficient 2p · Q / (Q + 1) or the coefficient −2p · Q / (Q + 1) by the pitch of the photoelectric conversion element. Positioning means for relatively positioning Imaging control means for imaging on the solid-state electronic image sensing device for each relative positioning position where the said positioning means each time,
An imaging apparatus comprising: Under the control of the imaging control means, a composite subject image signal representing a composite subject image is generated using a plurality of video signals obtained by imaging using the solid-state electronic imaging device for each relatively positioned position. Synthetic subject image signal generating means,
The imaging device according to claim 1, further comprising: Mode setting means for setting the shift imaging mode or the normal imaging mode,
An optical low-pass filter that can be disposed in front of the solid-state electronic image sensor and retractable from the front of the solid-state electronic image sensor;
When the shifted imaging mode is set by the mode setting means, the optical low-pass filter is retracted from the front of the solid-state electronic imaging device, and the normal imaging mode is set by the mode setting means. Optical low-pass filter control means for disposing the optical low-pass filter in front of the solid-state electronic image sensor, and the solid-state electronic image sensor by the positioning means when the normal imaging mode is set by the mode setting means Positioning stop means for stopping relative positioning of
The imaging apparatus according to any one of claims 1 to 3, further comprising: The aperture diameter of the light receiving surface of the photoelectric conversion element is not less than a value obtained by multiplying the pitch of the photoelectric conversion element by 1 / (N + 1).
The imaging device according to claim 1. The aperture diameter of the light receiving surface of the photoelectric conversion element is a value obtained by multiplying the pitch of the photoelectric conversion element by a value of 2 / (N + 1) or less.
The imaging device according to claim 1. The aperture diameter of the light receiving surface of the photoelectric conversion element is a value obtained by multiplying the pitch of the photoelectric conversion elements by 2 (N + 1).
The imaging device according to claim 1. The aperture diameter of the light receiving surface of the photoelectric conversion element is not less than a value obtained by multiplying the pitch of the photoelectric conversion element by 1 / (Q + 1).
The imaging device according to claim 2. The aperture diameter of the light receiving surface of the photoelectric conversion element is a value obtained by multiplying the pitch of the photoelectric conversion element by a value of 2 / (Q + 1) or less.
The imaging device according to claim 2. The aperture diameter of the light receiving surface of the photoelectric conversion element is a value obtained by multiplying the pitch of the photoelectric conversion element by 2 / (Q + 1).
The imaging device according to claim 2. The composite subject image signal generation means includes:
A memory for storing a plurality of video signals obtained by imaging using the solid-state electronic image sensor at each relatively shifted position under the control of the imaging control means as a synthesized subject image signal representing a synthesized subject image Is,
The imaging device according to claim 3. The exposure time of the solid-state electronic imaging device is the same for each position relatively positioned by the positioning means;
The imaging device according to any one of claims 1 to 11. An amplifying circuit for amplifying each video signal of a plurality of video signals obtained by imaging using the solid-state electronic imaging device for each position relatively positioned by the positioning means, at the same amplification factor;
The imaging device according to any one of claims 1 to 12, further comprising: An amplification circuit for amplifying each video signal of a video signal obtained by imaging using the solid-state electronic image sensor at each position relatively positioned by the positioning means, and a video signal amplified by the amplification circuit Amplification control means for controlling amplification in the amplification circuit in accordance with the exposure time of the solid-state electronic imaging device at each position relatively positioned by the positioning means so that the levels are equal;
The imaging device according to any one of claims 1 to 11, further comprising: The positioning means positions the solid-state electronic image sensor;
The imaging device according to any one of claims 1 to 14. An imaging lens provided in front of the solid-state electronic image sensor and movable in a direction perpendicular to the central axis of the solid-state electronic image sensor;
The positioning means positions the imaging lens;
The imaging device according to any one of claims 1 to 14. A holding base for holding the imaging device so as to be movable in a direction perpendicular to the optical axis of the solid-state electronic imaging device;
The positioning means positions the imaging device;
The imaging device according to any one of claims 1 to 14. The color filter array is a Bayer array,
The imaging device according to claim 1. Mode setting means for setting the shift imaging mode or the normal imaging mode,
A mechanical shutter that is disposed in front of the solid-state electronic image sensor and that performs an operation of exposing the solid-state electronic image sensor for an exposure time corresponding to a shutter speed;
The shifted imaging mode is set by the electronic shutter means for causing the electronic shutter to output the signal charge accumulated during the exposure time according to the shutter speed as the video signal from the solid-state electronic imaging device, and the mode setting means. Shutter control means for causing the electronic shutter to perform an electronic shutter, and for the normal imaging mode to be set by the mode setting means to perform an operation using the mechanical shutter;
The imaging device according to claim 1, further comprising: A mechanical shutter disposed in front of the solid-state electronic image sensor;
The imaging control means includes
With the mechanical shutter opened, the solid-state electronic image pickup device picks up an image at each position relatively positioned by the positioning means, and the solid-state electronic imaging at each position relatively positioned by the positioning means. After imaging the device, the solid-state electronic imaging device is imaged with the mechanical shutter closed.
Each of the plurality of video signals obtained by causing the solid-state electronic image pickup device to pick up an image with the mechanical shutter opened is picked up by the solid-state electronic image pickup device with the mechanical shutter closed. Black level correction means for correcting the black level using the reference video signal obtained by the processing,
The imaging device according to claim 1, further comprising: Set the shifting imaging mode, shifting imaging mode setting means,
Image stabilization mode setting means for setting image stabilization mode,
An image pickup lens provided in front of the solid-state electronic image pickup device and movable in a direction perpendicular to a central axis of the solid-state electronic image pickup device, and a camera shake correction mode set by the camera shake correction mode setting means. , Further comprising camera shake correction means for performing camera shake correction by moving the imaging lens in a direction perpendicular to the central axis of the solid-state electronic image sensor,
In response to the shift imaging mode being set by the shift imaging mode setting means, the positioning means positions the solid-state electronic image sensor at a position relatively positioned by the positioning means.
The imaging device according to any one of claims 1 to 14. A plurality of photoelectric conversion elements are arranged in a row direction and a column direction, respectively, and a color filter having a characteristic of transmitting a specific color component is arranged on the light receiving surface of the photoelectric conversion element in the row direction. And the signal charge accumulated in the photoelectric conversion element is converted into a video signal representing the subject image and output by imaging the subject.
The positioning means relatively positions the solid-state electronic image sensor in the row direction and the column direction, and the distance between the N photoelectric conversion elements whose color filter repetition intervals are even numbers in the row direction and the column direction, respectively. Is obtained by multiplying the coefficient m · N / (N + 1) by multiplying all m, which are natural numbers including 0 of N / 2 or less, by N / (N + 1), by the pitch of the photoelectric conversion element. The number of all the positions that can be obtained and all m that are natural numbers including 0 of N / 2 or less are negative. The coefficient obtained by multiplying N / (N + 1) by m / N / (N + 1) is the pitch of the photoelectric conversion element. Relatively position the solid-state electronic image sensor at all positions obtained by multiplying
The imaging control means causes the solid-state electronic imaging device to take an image for each position relatively positioned by the positioning means.
Control method of imaging apparatus. A plurality of photoelectric conversion elements are arranged in a row direction and a column direction, respectively, and a color filter having a characteristic of transmitting a specific color component is arranged on the light receiving surface of the photoelectric conversion element in the row direction. And the signal charge accumulated in the photoelectric conversion element is converted into a video signal representing the subject image and output by imaging the subject.
The positioning means relatively positions the solid-state electronic image sensor in the row direction and the column direction, and the distance between the Q photoelectric conversion elements whose color filter repeat intervals are odd numbers in the row direction and the column direction, respectively. , The coefficient p · Q / (Q + 1) obtained by multiplying all p, which are natural numbers including 0 less than Q / 2 by Q / (Q + 1), is multiplied by the pitch of the photoelectric conversion element. All the obtained positions and all ps which are natural numbers including 0 of Q / 2 or less are negative. -P · Q / (Q + 1) is a coefficient obtained by multiplying Q / (Q + 1) by -p · Q / (Q + 1). All the positions obtained by multiplying the pitch and all the positions obtained by multiplying the coefficient 2p · Q / (Q + 1) or the coefficient −2p · Q / (Q + 1) by the pitch of the photoelectric conversion element Relatively position the image sensor,
The imaging control means causes the solid-state electronic imaging device to take an image for each position relatively positioned by the positioning means.
Control method of imaging apparatus.

Images