JP2013098792A - Imaging device and control method for imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device and a control method for the imaging device that enable appropriate main photographing both in a global reset shutter system and in a rolling shutter system without complicating the configuration.SOLUTION: In performing main photographing using a rolling shutter system in which a charge reset operation is performed for each pixel line of a CMOS sensor, speed of reading out an image signal from the CMOS sensor is made higher than in performing main photographing using a global reset shutter system in which a charge reset operation is performed on all pixels of the CMOS sensor simultaneously.

Description

  The present invention relates to an imaging apparatus and a method for controlling the imaging apparatus.

  In recent years, an image pickup apparatus such as a digital camera that electronically records an image is increasingly equipped with a CMOS type image pickup device. As a driving method of a CMOS type image pickup device mounted on a digital camera, a rolling shutter method and a global reset shutter method are known.

  The rolling shutter system is a system that performs a charge reset operation for each pixel line (horizontal line) of the image sensor. In the rolling shutter method, exposure and readout are also performed for each pixel line, and the timing of exposure and readout varies from pixel line to pixel line. Therefore, especially when the subject is moving at high speed, the image is likely to be distorted. Known to be susceptible to camera shake.

  On the other hand, the global reset shutter method is a method in which the charge reset operation is performed simultaneously on all the pixels of the image sensor. In the case of the global reset shutter method, since all pixels start exposure at the same time, image distortion does not occur. However, since reading is performed sequentially from the upper pixel line of the image, the lower pixel line at the time of reading advances. The image gets brighter as it goes down. Therefore, when photographing with the global reset shutter system, it is common to prevent exposure from proceeding at the time of reading by using a mechanical shutter that mechanically blocks light incident on the image sensor.

  The rolling shutter method is not suitable for actual recording for recording, which requires high-resolution image capturing, because the larger the number of pixel lines read from the image sensor, the greater the effect of image distortion and camera shake as described above. Has been. For this reason, a live view image (through image) for monitoring is acquired by the rolling shutter method, and the driving method of the image sensor is switched from the rolling shutter method to the global reset shutter method at the time of actual photographing, and the main image is obtained by the global reset shutter method. I often shoot. A live view image is usually read out by thinning out pixel lines in order to ensure real-time performance, so that there is little influence of image distortion and camera shake due to the rolling shutter system.

However, since the rolling shutter system does not use a mechanical shutter, it has the following advantages over the global reset shutter system, and it is hoped that proper shooting can be performed by the rolling shutter system even in actual recording for recording. It is rare.
-Vibration shake due to mechanical shutter operation does not occur.
-Shutter sound does not occur, so it is suitable for shooting in a quiet place such as a concert hall or when you want to shoot a natural scene without being aware of the shooting.
-It is not necessary to ensure the precision of closing the mechanical shutter in order to realize a high-speed shutter, and a high-speed shutter can be easily realized simply by specifying the number and position of electronic shutters.

  With respect to an image pickup apparatus equipped with a CMOS type image pickup device, Patent Document 1 discloses that a through image of a subject obtained before actual photographing is analyzed, and one of a plurality of shutter drive modes is selected based on the analysis result. Is automatically selected and a CMOS image sensor is driven in the selected shutter drive mode.

  By the way, the image signal read from the image sensor is converted into digital image data, and various image processing is performed in a later stage LSI (signal processing IC). For this reason, the speed at which the image signal is read from the image sensor is normally determined at a speed at which the image signal read from the image sensor can be processed by the LSI without being missed, in accordance with the performance of the LSI at the subsequent stage. At this time, in the global reset shutter method, since reading and exposure are not performed simultaneously, there is no problem even if image signals are read at a speed that matches the performance of the LSI in the subsequent stage. On the other hand, in the rolling shutter method, reading of the upper pixel line of the image and exposure of the lower pixel line of the image are performed at the same time. Therefore, if the speed of reading the image signal from the image sensor is determined according to the performance of the LSI in the subsequent stage. The influence of the above-described image distortion and camera shake is large at the time of main photographing in which the number of pixel lines to be read is large. Therefore, in order to appropriately perform the main photographing with both the global reset shutter method and the rolling shutter method, it is required to perform reading suitable for each method.

  In the technique described in Patent Document 1 described above, the drive mode of the image sensor is automatically selected in accordance with the movement state of the subject, but reading in each drive mode is not considered. For this reason, it has a function to detect the motion vector of the subject by analyzing the through image and a function to detect the amount of camera shake based on the detection value of the gyro sensor. It is necessary to perform the main photographing by the method, and the configuration becomes complicated.

  The present invention has been made in view of the above, and there is provided an imaging apparatus and an imaging apparatus capable of appropriately performing main imaging with both the global reset shutter system and the rolling shutter system without complicating the configuration. It aims to provide a control method.

  In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention performs imaging in a first mode in which a charge reset operation is performed for each pixel line of the imaging element and the imaging element. 1st control means which makes the speed which reads an image signal from the above-mentioned image sensor faster than the case where it picturizes in the 2nd mode which performs charge reset operation simultaneously for all the pixels of the above-mentioned image sensor. .

  An imaging apparatus control method according to the present invention is an imaging apparatus control method comprising an imaging element and a control unit, wherein the control unit performs a charge reset operation for each pixel line of the imaging element. When shooting in the first mode, the speed at which an image signal is read out from the image sensor is faster than in the second mode in which charge reset operation is performed simultaneously on all pixels of the image sensor. And

  According to the present invention, when shooting is performed in the first mode (rolling shutter method) in which the charge reset operation is performed for each pixel line of the image sensor, the second mode in which the charge reset operation is simultaneously performed for all the pixels of the image sensor ( Since the speed at which the image signal is read out from the image sensor is faster than when shooting with the global reset shutter method), both the global reset shutter method and the rolling shutter method can be used without complicating the configuration. There is an effect that the main photographing can be performed appropriately.

FIG. 1-1 is a top view of the digital camera. FIG. 1-2 is a front view of the digital camera. FIG. 1-3 is a back view of the digital camera. FIG. 2 is a block diagram illustrating a hardware configuration of the digital camera. FIG. 3 is a diagram showing a shooting control sequence when shooting a still image by the global reset shutter method. FIG. 4 is a diagram illustrating a shooting control sequence when a still image is shot by the rolling shutter method. FIG. 5 is an image diagram illustrating an example of a processing flow in the signal processing IC during still image shooting (main shooting). FIG. 6 is an image diagram showing another example of the processing flow in the signal processing IC during still image shooting (main shooting). FIG. 7 is an image diagram showing an example of a processing flow in the signal processing IC when continuous shooting is performed by the rolling shutter method. FIG. 8 is an image diagram showing another example of the processing flow in the signal processing IC when continuous shooting is performed by the rolling shutter method. FIG. 9 is a diagram illustrating a shooting control sequence when a still image is shot with long exposure using the rolling shutter method.

  Exemplary embodiments of an imaging apparatus and an imaging apparatus control method according to the present invention will be described below in detail with reference to the accompanying drawings. The embodiment described below is an example in which the present invention is applied to a digital camera.

<First Embodiment>
FIGS. 1-1 to 1-3 are views showing the external appearance of a digital camera according to the present embodiment, FIG. 1-1 is a top view of the digital camera, and FIG. 1-2 is a front view of the digital camera. FIG. 1C is a back view of the digital camera.

  As shown in FIG. 1A, the digital camera according to the present embodiment has a sub LCD 1 that displays the number of still images that can be taken, and a release shutter button that is pressed during still image shooting. 2 and a mode dial 3 that is operated when switching various modes such as a recording (photographing) mode for recording an image, a reproduction mode for reproducing the recorded image, and a SETUP mode for camera setting.

  Also, as shown in FIG. 1-2, the digital camera according to the present embodiment has a strobe light emitting unit 4 that emits a strobe, a distance measuring unit 5 that measures the distance to the subject, and a remote controller (not shown). A remote control light receiving unit 6 that receives an infrared signal from a terminal, a lens barrel unit 7 including optical members such as a zoom lens, a focus lens, and a mechanical shutter, and an optical viewfinder (front) 8 are provided. Further, the side surface of the digital camera according to the present embodiment is provided with a memory card throttle for inserting a memory card and a battery accommodating portion for accommodating a battery. The memory card throttle and the battery accommodating portion are closed by a lid 9. Has been.

  In addition, the digital camera according to the present embodiment has an autofocus LED (AFLED) 10 that is turned on when the autofocus is activated, a strobe LED 11 that is turned on when the strobe is lighted, a display of various setting screens, a display of a reproduced image, and the like. An LCD monitor 12 used as an electronic viewfinder at the time of shooting, an optical viewfinder (back surface) 13, a zoom button (WIDE) 14 operated during wide-angle zooming, and a zoom button (TELE) 15 operated during telephoto zooming, It has a power switch 16 and a self-timer / deletion switch 17 that is operated when the self-timer is activated.

  Furthermore, the digital camera according to the present embodiment displays on the LCD monitor 12 the menu switch 18 that is operated when the menu is selected, the OK switch 19 that is operated when the selected item is confirmed, and the LCD monitor 12. An image confirmation switch (left) 20 and an image confirmation switch (right) 21 that are operated when switching the playback image, a macro switch 22 that is operated when performing macro photography, and a flash that is operated when switching the flash mode. It has a switch 23 and a display switch 24 operated when switching the display of the LCD monitor 12.

  FIG. 2 is a block diagram showing a hardware configuration of the digital camera according to the present embodiment. As shown in FIG. 2, the digital camera according to the present embodiment includes a front end IC 30, a signal processing IC 40, an SDRAM 50, a ROM 60, a clock generation unit 70, and a motor driver 80. The LCD monitor 12 and the operation unit 90 of the digital camera described above are connected to the signal processing IC 40. The operation unit 90 is a general term for various buttons and switches (see FIGS. 1-1 to 1-3) for operating the digital camera.

  The clock generation unit 70 generates a clock that is a reference for the operation of the digital camera. The clock generated by the clock generation unit 70 is supplied to the front end IC 30 and the signal processing IC 40.

  The motor driver 80 drives optical members such as the zoom lens 81, the focus lens 82, and the mechanical shutter 83 in accordance with a command from the CPU 100 in the signal processing IC 40 described later.

  The SDRAM 50 has a function as an image memory for temporarily storing digital image data processed by the signal processing IC 40 and a function as a work area for the CPU 100 in the signal processing IC 40 to execute a program. The ROM 60 stores a program and control data for controlling the digital camera.

  The front-end IC 30 includes a CMOS-type image sensor (hereinafter referred to as a CMOS sensor) 31, a CDS 32, an AGC 33, and an A / D 34. The analog image signal read out from the CMOS sensor 31 is subjected to correlated double sampling in the CDS 32, then gain adjusted in the AGC 33, converted into digital image data in the A / D 34, and input to the signal processing IC 40. Is done.

  The signal processing IC 40 includes a sensor I / F unit 41, a memory controller 42, a camera I / F unit 43, an image processing unit 44, a compression / decompression unit 45, a display output control unit 46, and a media I / F. A unit 47, a horizontal / vertical synchronization signal generation unit 48, and a CPU 100 are included.

  The sensor I / F unit 41 inputs the digital image data (RAW-DATA) output from the front end IC 30 into the signal processing IC 40.

  The memory controller 42 controls writing and reading of data with respect to the SDRAM 50.

  The camera I / F unit 43 performs image processing such as optical correction (black level correction, defect pixel correction, shading correction, etc.) and detection processing on the RAW-DATA input by the sensor I / F unit 41. Do. In this specification, the image processing performed on the RAW-DATA by the camera I / F unit 43 is referred to as first image processing.

  The image processing unit 44 performs white balance gain multiplication, gamma correction, and RGB interpolation processing (Bayer interpolation) on the RAW-DATA that has been subjected to the first image processing and temporarily written in the SDRAM 50, and further performs edge enhancement. And color setting, etc., and conversion into YUV image data which is a luminance / color difference signal. Furthermore, the image processing unit 44 performs a resizing process on the converted YUV image data to change the image size in accordance with the display or recording size. In this specification, the image processing performed by the image processing unit 44 is referred to as second image processing.

  The compression / decompression unit 45 compresses and decompresses the YUV image data generated by the image processing unit 44 according to a predetermined format (for example, JPEG).

  The display output control unit 46 controls the output of display data to the LCD monitor 12 and an external display device (not shown). Specifically, the display output control unit 46 generates display data by combining the YUV image data stored in the SDRAM 50 and OSD (on-screen display) data for displaying a shooting mode icon or the like, A signal such as a synchronization signal is added and output to the LCD monitor 12 or an external display device.

  The media I / F unit 47 controls writing and reading of data to and from the memory card.

  The horizontal / vertical synchronization signal generator 48 generates a horizontal synchronization signal HD and a vertical synchronization signal VD in response to a command from the CPU 100. The horizontal synchronization signal HD is a synchronization signal serving as a reference for processing one horizontal line of an image (one pixel line of the CMOS sensor 31), and the vertical synchronization signal VD is a synchronization signal serving as a reference for processing one frame of an image. It is. The horizontal synchronization signal HD and the vertical synchronization signal VD generated by the horizontal / vertical synchronization signal generation unit 48 are supplied to the front end IC 30 and the sensor I / F unit 41. Each part of the front end IC 30 operates based on the horizontal synchronization signal HD and the vertical synchronization signal VD, and the sensor I / F unit 41 synchronizes with the operation of the front end IC 30 based on the horizontal synchronization signal HD and the vertical synchronization signal VD. Input digital image data.

  The CPU 100 loads a program and control data for controlling the digital camera stored in the ROM 60 onto the SDRAM 50, and comprehensively controls the operation of the entire digital camera based on the program code. That is, based on an instruction from the operation unit 90 or an operation instruction from a remote control terminal (not shown), the CPU 100 can appropriately capture, display, and record an image corresponding to the instruction. Each processing unit in the processing IC 40 is controlled.

  In particular, in the digital camera according to the present embodiment, when the CPU 100 performs the main shooting with the rolling shutter method (first mode), the CPU 100 performs the main shooting with the global reset shutter method (second mode). It has a function (first control means) for increasing the speed of reading image signals from the CMOS sensor 31. Note that the selection of whether to perform the main shooting by the rolling shutter method or the main shooting by the global reset shutter method may be performed based on, for example, a setting by the photographer, or the digital camera may perform the shooting scene. It is also possible to automatically perform the determination.

  The rolling shutter system is a system that performs a charge reset operation for each pixel line (horizontal line) of the CMOS sensor 31. The global reset shutter method is a method in which a charge reset operation is performed simultaneously on all pixels of the CMOS sensor 31. In the global reset shutter method, since the image signal is read from the CMOS sensor 31 with the mechanical shutter 83 closed, there is no influence on the image quality even if the reading takes time. On the other hand, in the case of the rolling shutter method, exposure and readout are performed for each pixel line, and readout of one pixel line and exposure of another pixel line are performed at the same time. This increases the influence of image quality and causes deterioration in image quality. Therefore, in the digital camera according to the present embodiment, when performing the main shooting by the rolling shutter method, the image signal is read from the CMOS sensor 31 faster than the global reset shutter method, thereby causing image distortion and camera shake. Therefore, even with the rolling shutter system, a high-quality image can be taken.

  FIG. 3 is a diagram showing a shooting control sequence when shooting a still image by the global reset shutter method. In the figure, RL indicates the timing when the release shutter button 2 is pressed. The frame period when the release shutter button 2 is pressed is the V1 period, and is started by the first vertical synchronization signal VD after the release shutter button 2 is pressed. The frame period to be performed is referred to as a V2 period, and the frame periods started by the subsequent vertical synchronization signal VD are sequentially referred to as a V3 period, a V4 period, a V5 period,. The frame period is a period divided by the vertical synchronization signal VD generated by the horizontal / vertical synchronization signal generator 48. In addition, each hatched area in the figure represents one image, and the top and bottom of the hatched area correspond to the top and bottom of the pixel line (horizontal line) of the CMOS sensor 31.

  In the digital camera according to the present embodiment, when the power switch 16 is turned on while the shooting mode is selected with the mode dial 3, the CMOS sensor 31 is activated in the image monitor mode. In the image monitor mode, the live view image is acquired by the rolling shutter method, and the live view image is displayed on the LCD monitor 12. Since the live view image does not need to be a high-definition image, the image signal is read from the CMOS sensor 31 in a state where pixel lines are thinned out to about 1/3, for example. The photographer confirms the state of the subject while viewing the live view image displayed on the LCD monitor 12, determines the composition, and presses the release shutter button 2 at a desired timing. At this time, when the release shutter button 2 has a two-stage switch of half-press and full-press, when the release shutter button 2 is half-pressed, AF (autofocus) processing is performed. In the AF process, for example, hill-climbing AF for finding the maximum contrast value while moving the focus lens 82 is performed.

  Thereafter, when the release shutter button 2 is fully pressed, the exposure settings (electronic shutter setting, AGC setting, aperture setting, etc.) of the still image to be taken are set during the V1 period after the release shutter button 2 is pressed. Done. The electronic shutter is set by writing a specified set value (number of electronic shutters) in accordance with a command from the CPU 100. When the first vertical synchronization signal VD after the release shutter button 2 is pressed rises and the V2 period starts, the CPU 100 switches from the image monitor mode to the still image shooting mode, and the CMOS sensor using the global reset shutter method. The driving of 31 is started. At this time, a timer for controlling the timing for closing the mechanical shutter 83 is started.

  Then, at the timing when the time determined by the electronic shutter elapses from the start of the V2 period, the process of resetting the charges at the same time for all the pixels of the CMOS sensor 31 is performed, and the exposure of the CMOS sensor 31 is started at the same time. After that, the mechanical shutter 83 is closed at the timing when a predetermined time has elapsed by the timer started with the start of the V2 period, and the exposure ends. That is, in the global reset shutter system, the exposure time of the CMOS sensor 31 is determined by using the electronic shutter and the mechanical shutter 83 in combination, and all pixels are exposed simultaneously during the V2 period.

  Then, when the next vertical synchronizing signal VD rises and the V3 period starts, reading is sequentially performed from the upper pixel line of the CMOS sensor 31. At this time, the mechanical shutter 83 is in a closed state, and no light is incident on the CMOS sensor 31, so that exposure of pixels does not proceed during the V3 period, and an appropriate image can be read out. In the global reset shutter method, the exposure and readout of the CMOS sensor 31 are independently performed in different frame periods as described above. When reading of all pixel lines from the CMOS sensor 31 is completed, the live view image is acquired again by the rolling shutter method in the subsequent frame period (V4 period, V5 period,...).

  FIG. 4 is a diagram illustrating a shooting control sequence when a still image is shot by the rolling shutter method. Similarly to FIG. 3, RL in the figure indicates the timing when the release shutter button 2 is pressed, the frame period when the release shutter button 2 is pressed is the V1 period, and the first after the release shutter button 2 is pressed. A frame period started by the vertical synchronization signal VD is set as a V2 period, and thereafter, a frame period started by the vertical synchronization signal VD is set as a V3 period, a V4 period, a V5 period,. Each hatched area in the figure represents one image, and the top and bottom of the hatched area correspond to the top and bottom of the pixel line (horizontal line) of the CMOS sensor 31.

  In the V1 period, the CMOS sensor 31 operates in the image monitor mode as in the case of taking a still image by the global reset shutter method, and the live view image is acquired by the rolling shutter method. When the first vertical synchronization signal VD after the release shutter button 2 is pressed rises and the V2 period starts, the CPU 100 switches from the image monitor mode to the still image shooting mode. In the example of FIG. 4, since a still image is shot by the rolling shutter method, the driving method of the CMOS sensor 31 remains the rolling shutter method even when the image monitor mode is switched to the still image shooting mode. However, since it is necessary to acquire a higher-definition image in the still image shooting mode than in the image monitor mode, reading is performed on all the pixel lines of the CMOS sensor 31.

  Then, exposure of a still image by the rolling shutter method is started in the V2 period, and reading is performed in the next V3 period. In the case of the rolling shutter system, the charge reset operation is generated for each pixel line, and the process of resetting the charge from the upper pixel line of the CMOS sensor 31 is sequentially performed. The exposure time is determined only by the electronic shutter, and reading is performed in order from the upper pixel line after the exposure is completed. For this reason, the exposure timing is different for each pixel line, and one image is exposed over two frame periods of the V2 period and the V3 period.

  The slope of the hatched area in the period V3 in FIG. 4 corresponds to the reading speed when a still image is shot by the rolling shutter method, and until the reading of the next pixel line is started after reading one pixel line. Can be defined in terms of time. The time from reading one pixel line to starting reading the next pixel line depends on the cycle (HD period) in which the horizontal synchronization signal HD is output, and the number of clocks constituting the HD period and the CMOS sensor 31 driving frequency. Therefore, the reading speed can be changed by changing the number of clocks constituting the HD period.

  In the digital camera according to the present embodiment, when a still image is captured by the rolling shutter method, the HD period is configured in the V3 period in which the still image is read, compared to when the still image is captured by the global reset shutter method. Reading speed is increased by reducing the number of clocks. Such control can be realized by the CPU 100 issuing a command designating the number of clocks constituting the HD period to the horizontal / vertical synchronization signal generation unit 48 that generates the horizontal synchronization signal HD. However, since the minimum HD period (mainly composed of a blanking period and a required readout time) is determined in each of the image monitor mode and the still image shooting mode, the CPU 100 determines that the HD period is in the still image shooting mode. The reading speed is increased by reducing the number of clocks constituting the HD period within a range not shorter than the minimum HD period.

  When reading of all the pixel lines from the CMOS sensor 31 is completed, switching from the still image shooting mode to the image monitor mode is performed, and in the subsequent frame period (V4 period, V5 period...), The rolling shutter is again performed. The live view image is acquired by the method. In the rolling shutter method, one image is exposed over two frame periods. However, switching from the still image shooting mode to the image monitor mode is performed between the V3 period and the V4 period in FIG. As a result, proper exposure cannot be performed. For this reason, the image exposed across the V3 period and the V4 period becomes a discarded frame, and is displayed on the LCD monitor 12 from the live view image read in the next V5 period.

  As described above with reference to specific examples, the digital camera according to the present embodiment has a speed for reading an image signal from the CMOS sensor 31 when shooting a still image (main shooting) by the rolling shutter method. Since it is faster than the global reset shutter method, it effectively suppresses the effects of image distortion and camera shake, which are the disadvantages of the rolling shutter method, and appropriately captures still images using the rolling shutter method. Can do. Further, when taking a still image (main shooting) by the global shutter method, the image signal can be read from the CMOS sensor 31 at a relatively low speed in accordance with the processing performance of the signal processing IC 40 in the subsequent stage. The increase in cost and the increase in power consumption due to the high performance of the IC 40 can be effectively suppressed. As described above, according to the digital camera according to the present embodiment, it is possible to appropriately perform the main photographing with both the global reset shutter method and the rolling shutter method without complicating the configuration.

<Second Embodiment>
Next, a second embodiment will be described. The digital camera according to the second embodiment is implemented in the signal processing IC 40 while the digital image data (RAW-DATA) output from the front end IC 30 is being written in the SDRAM 50 when performing the main shooting by the rolling shutter method. Of the image processing to be performed, the processing speed of at least a part of the image processing is made slower than that in the case of performing the main photographing by the global reset shutter method. Since the outline of the hardware configuration and operation of the digital camera is the same as that of the first embodiment, only the features of this embodiment will be described below, and the description overlapping with the first embodiment will be omitted as appropriate. To do.

  In the digital camera according to the present embodiment, when the CPU 100 performs the main photographing by the rolling shutter method, the function of increasing the speed of reading the image signal from the CMOS sensor 31 compared to the case of the global reset shutter method (first control means) ), The function of slowing down the processing speed of at least some of the image processing performed in the signal processing IC 40 while the digital image data (RAW-DATA) output from the front-end IC 30 is being written in the SDRAM 50 (first operation). 2 control means).

  In order to read an image signal from the CMOS sensor 31 at a high speed, it is necessary that the digital image data output at a high output rate from the front-end IC 30 can be written to the SDRAM 50 without being missed by the signal processing IC 40 in the subsequent stage. The load on the data line (bus) between the IC 40 and the SDRAM 50 increases. At this time, particularly when continuous shooting or the like is performed, image processing for the digital image data of the previous frame and processing for writing the digital image data of the subsequent frame to the SDRAM 50 are performed in parallel. Since the bandwidth of the bus that can be used for writing image data is limited, it is difficult to read out the image signal from the CMOS sensor 31 at high speed. Further, not only in continuous shooting, but also when processing is performed for each data of a predetermined number of lines for an image of one frame, writing to the SDRAM 50 and image processing are performed in parallel, and the same problem occurs. Arise.

  As a method of solving the above problem, it is conceivable to review the architecture of the digital camera to increase the speed of the signal processing IC 40 and the SDRAM 50 and improve the performance of the bus between them. However, such a correspondence leads to a significant cost increase of the digital camera and results in an increase in power consumption.

  Therefore, in the digital camera according to the present embodiment, when the main shooting is performed by the rolling shutter method, the digital image data (RAW-DATA) output from the front end IC 30 is written in the SDRAM 50 in the signal processing IC 40. The above problem is solved by slowing down the processing speed of at least a part of the image processing. Note that the processing speed of the image processing is not necessarily reduced during the entire period in which the RAW-DATA output from the front-end IC 30 is written in the SDRAM 50, while the RAW-DATA is being written in the SDRAM 50. In some of the periods, the image processing speed may be reduced.

  The processing speed of the image processing performed in the signal processing IC 40 is determined according to the image processing clock set by the instruction of the CPU 100. In the digital camera according to the present embodiment, the CPU 100 temporarily delays the image processing clock during at least a part of the period during which RAW-DATA is written to the SDRAM 50 when performing the main photographing by the rolling shutter method. . As a result, the data line (bus) between the signal processing IC 40 and the SDRAM 50 can be preferentially assigned to the RAW-DATA writing, and as a result, the image signal can be received from the CMOS sensor 31 without increasing the bus performance. Reading at high speed becomes possible.

  FIG. 5 is an image diagram illustrating an example of a processing flow in the signal processing IC 40 during still image shooting (main shooting). Each block in the figure corresponds to each processing block in the signal processing IC 40 shown in FIG. 2, and indicates that processing is performed in order from the left to the right in the figure. Note that data is written to and read from the SDRAM 50 via the memory controller 42, but the memory controller 42 is not shown in FIG.

  Digital image data (RAW-DATA) output from the front end IC 30 is taken into the signal processing IC 40 by the sensor I / F unit 41. The RAW-DATA captured in the signal processing IC 40 is transferred from the sensor I / F unit 41 to the camera I / F unit 43 via the memory controller 42, and the camera I / F unit 43 performs the first image processing. Done. Then, the RAW-DATA subjected to the first image processing is written into the SDRAM 50 by the memory controller 42.

  The RAW-DATA stored in the SDRAM 50 is read from the SDRAM 50 by the memory controller 42 and passed to the image processing unit 44. Then, after the second image processing is performed by the image processing unit 44, it is written again into the SDRAM 50 by the memory controller 42 as YUV image data.

  The YUV image data stored in the SDRAM 50 is read from the SDRAM 50 by the memory controller 42 and passed to the compression / decompression unit 45. Then, the compression / decompression unit 45 compresses the YUV image data in, for example, the JPEG format, and the JPEG image data is written again into the SDRAM 50 by the memory controller 42.

  The JPEG image data stored in the SDRAM 50 is read from the SDRAM 50 by the memory controller 42 and passed to the media I / F unit 47. Then, the media I / F unit 47 records JPEG image data on the memory card.

  In the example shown in FIG. 5, for example, when multiple frames of digital image data are processed simultaneously by continuous shooting, or when processing is performed for each data of a predetermined number of lines for an image of one frame, After the RAW-DATA that has undergone image processing is written to the SDRAM 50, the RAW-DATA that has undergone the first image processing is read from the SDRAM 50, the second image processing is performed by the image processing unit 44, and then the YUV The process of rewriting the image data in the SDRAM 50 conflicts. Therefore, when performing the main photographing by the rolling shutter method, the processing speed of the second image processing by the image processing unit 44 is set lower than that when performing the main photographing by the global reset shutter method, and the first image processing is performed. By giving priority to the process of writing the received RAW-DATA to the SDRAM 50, the image signal can be read from the CMOS sensor 31 at high speed without improving the bus performance. If the image signal can be read out from the CMOS sensor 31 at a high speed, as described in the first embodiment, it is possible to suppress the effects of image distortion and camera shake, which are the disadvantages of the rolling shutter system, and a high-quality image. Can be taken. Note that in the case of performing the main shooting by the global reset shutter method, the image quality is not affected even when the image signal is read out at a low speed, so the reading speed is set according to the load of the image processing performed in the signal processing IC 40 in the subsequent stage. What is necessary is just to determine, and it is not necessary to slow down the processing speed of image processing.

  FIG. 6 is an image diagram illustrating another example of a processing flow in the signal processing IC 40 during still image shooting (main shooting). As in FIG. 5, each block in the figure corresponds to each processing block in the signal processing IC 40 shown in FIG. 2, and indicates that processing is performed in order from the left to the right in the figure.

  The example shown in FIG. 6 is different from the example shown in FIG. 5 in the process of writing RAW-DATA to the SDRAM 50. That is, in the example of FIG. 5, the first image processing by the camera I / F unit 43 is performed on the RAW-DATA captured in the signal processing IC 40 by the sensor I / F unit 41, and then the SDRAM 50 is processed. I am writing. On the other hand, in the example of FIG. 6, RAW-DATA fetched into the signal processing IC 40 by the sensor I / F unit 41 is directly written in the SDRAM 50, and then, for RAW-DATA read from the SDRAM 50, The first image processing by the camera I / F unit 43 is performed, and the RAW-DATA subjected to the first image processing is written into the SDRAM 50 again. Other processes are the same as those in the example of FIG.

  In the example of FIG. 6, in contrast to the process of writing RAW-DATA to the SDRAM 50, the process of reading the RAW-DATA from the SDRAM 50, performing the first image processing by the camera I / F unit 43, and rewriting to the SDRAM 50, The RAW-DATA that has been subjected to the image processing is read from the SDRAM 50, the second image processing by the image processing unit 44 is performed, and the processing is rewritten in the SDRAM 50. Therefore, in the case of performing the main photographing by the rolling shutter method, the global reset shutter method uses the processing speed of the first image processing by the camera I / F unit 43 and the processing speed of the second image processing by the image processing unit 44. Therefore, the image signal can be read out from the CMOS sensor 31 at a high speed without increasing the bus performance by prioritizing the process of writing the RAW-DATA to the SDRAM 50, as compared with the case of performing the main photographing.

  In the example of FIG. 6, the RAW-DATA taken into the signal processing IC 40 by the sensor I / F unit 41 is written to the SDRAM 50 as it is, and the first by the camera I / F unit 43 while taking the RAW-DATA. There is no need to perform image processing. For this reason, compared with the example of FIG. 5 which performs 1st image processing, taking in RAW-DATA, the speed-up which reads an image signal from the CMOS sensor 31 is easy. However, in the example of FIG. 6, the number of times digital image data is read from and written to the SDRAM 50 increases, so that the total processing time increases compared to the example of FIG. 5. Therefore, it is desirable to apply the example of FIG. 6 only when the main photographing is performed by the rolling shutter method, and when the main photographing is performed by the global reset shutter method.

  As described above with reference to specific examples, the digital camera according to the present embodiment writes the RAW-DATA output from the front-end IC 30 to the SDRAM 50 when performing the main photographing by the rolling shutter method. Meanwhile, since the processing speed of at least a part of the image processing performed in the signal processing IC 40 is set to be slower than that in the case of performing the main photographing by the global reset shutter method, the CMOS sensor 31 is improved without improving the bus performance. Thus, the image signal can be read out at high speed, and the effects of image distortion and camera shake, which are disadvantages of the rolling shutter system, can be effectively suppressed.

<Third Embodiment>
Next, a third embodiment will be described. The digital camera according to the third embodiment is implemented in the signal processing IC 40 while the digital image data (RAW-DATA) output from the front-end IC 30 is being written in the SDRAM 50 in the case of performing the main photographing by the rolling shutter method. Among the image processing to be performed, at least a part of the image processing is stopped. Since the outline of the hardware configuration and operation of the digital camera is the same as that of the first embodiment, only the features of this embodiment will be described below, and the description overlapping with the first embodiment will be omitted as appropriate. To do.

  In the second embodiment, at the time of performing actual photographing by the rolling shutter method, at least a part of the image processing performed in the signal processing IC 40 while the RAW-DATA output from the front end IC 30 is being written in the SDRAM 50. By making the processing speed slower than the case where the main photographing is performed by the global reset shutter method, the processing for writing the RAW-DATA to the SDRAM 50 can be performed preferentially. As a result, the CMOS sensor 31 can be obtained without improving the bus performance. It has been explained that the image signal can be read from the camera at high speed. On the other hand, in the third embodiment, at the time of performing the main photographing by the rolling shutter method, at least one of the steps performed in the signal processing IC 40 while the RAW-DATA output from the front end IC 30 is being written in the SDRAM 50. By stopping the image processing of the part, the processing for writing RAW-DATA to the SDRAM 50 can be performed preferentially, and as a result, the image signal can be read from the CMOS sensor 31 at high speed without increasing the bus performance. I have to.

  In the digital camera according to the present embodiment, when the CPU 100 performs the main photographing by the rolling shutter method, the function of increasing the speed of reading the image signal from the CMOS sensor 31 compared to the case of the global reset shutter method (first control means) ) And a function (third control) for stopping at least a part of the image processing performed in the signal processing IC 40 while the digital image data (RAW-DATA) output from the front end IC 30 is being written to the SDRAM 50. Means). Note that the stop of the image processing is not necessarily performed during the entire period in which the RAW-DATA output from the front-end IC 30 is written in the SDRAM 50, but a part of the period during which the RAW-DATA is written in the SDRAM 50. In this case, the image processing may be stopped.

  The operation of the signal processing IC 40 is controlled by the CPU 100, and the timing at which the camera I / F unit 43 performs the first image processing and the timing at which the image processing unit 44 performs the second image processing are controlled by an instruction of the CPU 100. can do. In the digital camera according to the present embodiment, the CPU 100 stops at least a part of image processing during at least a part of a period during which RAW-DATA is written in the SDRAM 50 when performing the main photographing by the rolling shutter method. . As a result, the data line (bus) between the signal processing IC 40 and the SDRAM 50 can be occupied for RAW-DATA writing, and the speed at which the image signal is read from the CMOS sensor 31 is further increased than in the second embodiment. It becomes possible to increase the speed.

  FIG. 7 shows the digital camera according to the present embodiment, in which continuous shooting is performed by a rolling shutter system, and the processing of the previous frame (first captured image) and the processing of the rear frame (next captured image) are performed. It is an image figure which shows an example of the processing flow in signal processing IC40 in the case of performing in parallel, and respond | corresponds to the example of FIG.

  In the example of FIG. 7, while the camera I / F unit 43 writes the RAW-DATA of the subsequent frame to the SDRAM 50, the second image processing for the previous frame by the image processing unit 44 is stopped, and the RAW-DATA of the subsequent frame is stopped. Is written in the SDRAM 50, the second image processing for the previous frame by the image processing unit 44 is started.

  FIG. 8 shows an example of a processing flow in the signal processing IC 40 when continuous shooting is performed by the rolling shutter method in the digital camera according to the present embodiment, and the processing of the previous frame and the processing of the subsequent frame are performed in parallel. It is an image figure and corresponds to the example of FIG.

  In the example of FIG. 8, while the sensor I / F unit 41 writes the RAW-DATA of the subsequent frame to the SDRAM 50, the first image processing by the camera I / F 43 and the second image by the image processing unit 44 for the previous frame. The processing is stopped, and after the RAW-DATA of the subsequent frame is written in the SDRAM 50, the first image processing by the camera I / F 43 and the second image processing by the image processing unit 44 for the previous frame are sequentially started. ing.

  As described above with reference to specific examples, the digital camera according to the present embodiment writes the RAW-DATA output from the front-end IC 30 to the SDRAM 50 when performing the main photographing by the rolling shutter method. In the meantime, since at least a part of the image processing performed in the signal processing IC 40 is stopped, the speed at which the image signal is read from the CMOS sensor 31 is higher than that in the second embodiment without increasing the bus performance. Thus, it is possible to effectively suppress the effects of image distortion and camera shake, which are disadvantages of the rolling shutter system.

<Fourth Embodiment>
Next, a fourth embodiment will be described. In the digital camera according to the fourth embodiment, even when the main shooting is performed by the rolling shutter method, when the exposure time at the time of shooting is longer than a predetermined value, the speed at which the image signal is read from the CMOS sensor 31 is The speed is set to the same speed as in the case of actual photographing by the reset shutter method. Since the outline of the hardware configuration and operation of the digital camera is the same as that of the first embodiment, only the features of this embodiment will be described below, and the description overlapping with the first embodiment will be omitted as appropriate. To do.

  In the digital camera according to the first embodiment, when performing actual shooting by the rolling shutter method, the image signal is read from the CMOS sensor 31 faster than the global reset shutter method, thereby causing image distortion and camera shake. The effect of was small. However, even when the main shooting is performed by the rolling shutter method, if the shooting is performed for a long time exposure, even if the speed at which the image signal is read from the CMOS sensor 31 is increased, a great merit for the image quality cannot be obtained. . For example, if the time required to read an image signal from the CMOS sensor 31 is 200 msec, the difference in exposure start timing between the uppermost pixel line and the lowermost pixel line of the CMOS sensor 31 is 200 msec. Assuming that the exposure time is 5 sec, the difference in the exposure start timing is a very small value compared to the exposure time. Even if this value is further reduced by increasing the speed at which the image signal is read from the CMOS sensor 31, There is no expectation for the image quality.

  FIG. 9 is a diagram illustrating a shooting control sequence when a still image is shot with long exposure using the rolling shutter method. In the example of FIG. 9, exposure is started in order from the upper pixel line in the V2 period, exposure of all pixel lines is continued in the V3 period, and readout is sequentially performed from the upper pixel line in the V4 period. In the case of a longer exposure, after the V3 period is repeated according to the length of the exposure time, readout is sequentially performed from the upper pixel line in the V4 period.

  In this way, if the exposure time at the time of shooting is long, no merit for the image quality can be expected even if the speed at which the image signal is read from the CMOS sensor 31 is increased. Therefore, in the digital camera according to the present embodiment, when the CPU 100 performs the main shooting by the rolling shutter method, the exposure time at the time of shooting is compared with a predetermined value, and only when the exposure time is equal to or less than the predetermined value. If the speed at which the image signal is read from the CMOS sensor 31 is higher than that in the case of performing the main photographing by the global reset shutter method, and the exposure time is larger than a predetermined value, the speed at which the image signal is read from the CMOS sensor 31 is set to the global reset shutter method. Same as Note that the predetermined value used for the determination may be an optimal value according to the performance of the digital camera and the required image quality.

  As described above, the digital camera according to the present embodiment can read image signals from the CMOS sensor 31 when the exposure time at the time of shooting is longer than a predetermined value even when the main shooting is performed by the rolling shutter method. Is set to the same speed as the actual shooting by the global reset shutter method, so that the power consumption increases by increasing the reading speed more than necessary, or the image processing is stopped or processed as the reading speed increases. Problems such as an increase in processing time due to a slow speed can be effectively avoided.

  The embodiment of the present invention has been specifically described above, but the present invention is not limited to the above-described embodiment as it is, and is embodied by adding various modifications without departing from the scope of the invention in the implementation stage. can do. In other words, the configuration and operation of the digital camera described above are merely specific examples, and various modifications can be made according to the application and purpose.

30 Front-end IC
31 CMOS sensor 34 A / D
40 Signal processing IC
43 Camera I / F unit 44 Image processing unit 50 SDRAM
100 CPU

JP 2009-159594 A

Claims (6)

An image sensor;
When shooting is performed in the first mode in which the charge reset operation is performed for each pixel line of the image sensor, the imaging is performed more than in the case of shooting in the second mode in which the charge reset operation is performed simultaneously on all the pixels of the image sensor. An image pickup apparatus comprising: a first control unit that increases a speed of reading an image signal from the element. Conversion means for converting the image signal read from the image sensor into digital image data;
Storage means for storing the digital image data;
Image processing means for performing image processing on the digital image data;
When shooting in the first mode, the processing speed of the image processing by the image processing means is set to the second speed during at least a part of the period during which the digital image data is written in the storage means. The imaging apparatus according to claim 1, further comprising: a second control unit that is slower than when shooting in a mode. Conversion means for converting the image signal read from the image sensor into digital image data;
Storage means for storing the digital image data;
Image processing means for performing image processing on the digital image data;
When shooting in the first mode, a third operation is performed to stop at least a part of the image processing by the image processing unit during at least a part of the period during which the digital image data is written in the storage unit. The imaging apparatus according to claim 1, further comprising:   The third control unit starts image processing by the image processing unit after the writing of the digital image data to the storage unit is completed when shooting in the first mode. The imaging device according to claim 3.   Even when the first control unit performs shooting in the first mode, when the exposure time at the time of shooting is longer than a predetermined value, the second control unit sets a speed at which the image signal is read from the image sensor. The imaging apparatus according to any one of claims 1 to 4, wherein the speed is set to the same speed as when shooting in the mode. An imaging device control method comprising: an imaging element; and a control unit,
When the control unit performs shooting in the first mode in which the charge reset operation is performed for each pixel line of the image sensor, and in the second mode in which the charge reset operation is performed simultaneously for all the pixels of the image sensor. A method for controlling an imaging apparatus, characterized by increasing a speed at which an image signal is read from the imaging element.

Images