JP2021082994A - Imaging apparatus, control method of imaging apparatus - Google Patents

Abstract

To reduce power consumption of an imaging device capable of detecting a change in a photographic subject even when the change is frequently detected.SOLUTION: An imaging apparatus comprises an imaging device in which a plurality of pixel circuits are arranged in a matrix and constitute a block, the imaging device can output each of the added value of the pixel signal for each block and the pixel signal values of the pixel circuits. The imaging apparatus detects a photographic subject change on the basis of a difference between early added value of the image signal of the same block in the imaging device and the late added value, and when detecting the photographic subject change, the imaging apparatus outputs the pixel signal value of each of the pixel circuits included in the block in which the photographic subject change is detected, and does not output the pixel signal value of each of the pixel circuits included in the block in which the photographic subject change is not detected.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup apparatus and a control method for the image pickup apparatus.

Imaging devices are widely used for the purpose of observation, monitoring, and the like. Regarding the imaging operation by the imaging device, not only the imaging at a predetermined interval but also the subject (or the change of the subject) is detected by using the difference in the brightness value between the front and rear frames, and the judgment of the event occurrence due to the subject change is obtained. An imaging device that starts imaging and storage operations is known.

In Patent Document 1, subject detection is performed in a shooting drive mode different from that during normal shooting, and a subject is detected based on the output of a plurality of divided blocks to which pixel signals are added in the plane of the image sensor. Detect. As a result, the amount of information processing is reduced as image and luminance information to reduce power consumption.

Japanese Unexamined Patent Publication No. 2018-22935

However, in the above-mentioned prior art, the power consumption is the same as in the normal shooting mode in the shooting and recording operation after the subject is detected, so that the power consumption reduction effect is reduced when the subject is frequently detected.

Therefore, in an imaging device that detects a subject, we provide a technique that enables reduction of power consumption even when a change in the subject is frequently detected.

The invention for solving the above problem is an image pickup device in which pixel circuits are arranged in a matrix and can output an addition value of pixel signals for each block composed of a plurality of pixel circuits and a pixel signal value of each pixel circuit. An image sensor equipped with
A detection means for detecting a subject change based on the difference between the added values of the pixel signals output from the block at the same position in the image sensor and which are back and forth in time is provided.
When the detection means detects a subject change, the image sensor outputs the pixel signal value of each pixel circuit included in the block for the block in which the subject change is detected, and the block in which the subject change is not detected is output. The feature is that the pixel signal value of each pixel circuit included in the block is not output.

According to the present invention, in an imaging device that detects a subject, power consumption can be reduced even when a change in the subject is frequently detected.

The block diagram which shows the structural example of the image pickup apparatus 100 corresponding to an embodiment. Block diagram showing a configuration example of the image sensor 200 corresponding to the embodiment The figure which shows the structural example of the pixel array part 220 corresponding to an embodiment. The figure which shows the structural example of the pixel circuit 230 in embodiment. The figure which shows the configuration example of the event detection part 214 in embodiment. The figure explaining an example of the captured image and subject detection corresponding to an embodiment. The flowchart which shows an example of the operation of the image pickup apparatus 100 corresponding to an embodiment. The figure for demonstrating the structure of the image pickup element 200 corresponding to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate explanations are omitted.

[Embodiment 1]
Hereinafter, the first embodiment will be described with reference to the accompanying drawings. FIG. 1 is a block diagram showing a configuration example of an image pickup apparatus 100 corresponding to an example of a typical embodiment. In FIG. 1, the image pickup lens 101 collects light from a subject onto the image pickup element 200. The image sensor 200 is a CMOS image sensor, and is configured to be able to perform photoelectric conversion of light incident on the image sensor 101 and output it as an image signal. Further, the image pickup device 200 has a control function of selecting and changing the drive control of the image pickup device 200 itself according to the amount of change in the pixel output value and the detection result of the change region. By having the above function, it is possible to select / change to a suitable shooting mode from the detection of the change of the subject by the image sensor itself. Further, the image sensor 200 has a function of cropping to an arbitrary image pickup region and taking a picture.

The image processing unit 102 performs various corrections such as filter processing and digital image processing such as compression on the image signal output from the image sensor 200. Further, the pixel signal in the 4K mode, the FullHD mode, or the like is also subjected to image processing such as resizing according to each mode. The control unit 103 controls the drive timing of the image sensor 200, and also controls the overall drive and control of the entire image pickup device such as the image processing unit 102 and the display unit 106. Further, the control unit 103 receives a user's command and performs drive control according to a shooting mode selected from a plurality of shooting modes such as a still image mode, a FullHD mode, and a 4K mode.

Further, the image sensor 100 includes a detection mode in which the image sensor 200 selects and changes the shooting mode according to the detection of the subject change, and controls the operation permission of the image sensor 200 to the detection mode.

The memory 104 and the storage unit 105 are storage media such as a non-volatile memory and a memory card that store and hold the image signal output from the image processing unit 102. The display unit 106 is a display that displays captured images, various setting screens, and the like. The operation unit 107 is composed of a button, a touch panel, a switch, and the like, receives an operation input from the user, and reflects the user's command to the control unit 103.

FIG. 2 is a block diagram showing a configuration example of the image sensor 200 corresponding to a typical embodiment. The image sensor 200 includes a pixel array unit 220 in which a plurality of pixels are arranged in a matrix, and is composed of an output in pixel units and a set of a predetermined number of pixel circuits in the plane of the pixel array unit 220. It has a function of dividing into pixel blocks and adding and outputting pixel signals for each pixel block. The pixel signal output from the pixel array unit 220 is analog-digitally converted for each pixel row by the AD conversion circuit 212 or for each block row in pixel block units described later, and then sequentially events are driven by the horizontal scanning circuit 213. It is transferred to the detection unit 214.

The event detection unit 214 receives the control signal of the mode control unit 216, detects the amount of change in the pixel output value as a determination standard for a predetermined event with respect to the output in pixel block units, and outputs the information of the detection result to the image sensor. It is transmitted to the mode control unit 216 that controls the drive mode of the 200. Further, when the output of the pixel array unit 220 is in pixel units, the event detection unit 214 transfers the output pixel data as it is to the signal processing unit 215. Further, the event detection unit 214 integrates the output data in pixel block units and supplies the integrated data to the exposure control unit 217.

The signal processing unit 215 adds information data such as the amount of change in the pixel data and the drive mode of the image sensor 200 before and after the pixel data output from the event detection unit 214, and outputs the information data to the outside of the image sensor 200. The mode control unit 216 receives a signal from the event detection unit 214 inside the image sensor or the control unit 103 outside the image sensor, and drives the drive timing control signal to each of the AD conversion circuit 212, the horizontal scanning circuit 213, and the vertical scanning circuit 211. Is supplied, and drive control is performed according to each image pickup mode of the image pickup element 200. Further, the mode control unit 216 determines whether or not to drive the crop according to the signal from the event detection unit 214, and when driving the crop, controls the imaging region to be driven based on the signal from the event detection unit 214. And perform crop drive. Further, when the image sensor 200 is permitted to operate in the detection mode by the control unit 103, the mode control unit 216 sets the image sensor 200 as the detection mode and starts driving to add a pixel signal for each pixel block.

The vertical scanning circuit 211 performs line selection / driving in pixel units or pixel block units via signal lines connected for each line. The exposure control unit 217 calculates the exposure time as the exposure control of the image sensor based on the integrated data from the event detection unit 214, and supplies the exposure control signal of the image sensor to the mode control unit 216.

FIG. 3 is a diagram showing a configuration example of the pixel array unit 220 corresponding to a typical embodiment. A plurality of pixel circuits 230 are arranged in a matrix in the pixel array unit 220. Further, as an addition unit for event detection, a pixel block composed of a set of a plurality of pixel circuits is indicated by a dotted rectangular shape 240. In order to simplify the explanation, in the present embodiment, the case where the set of pixel circuits of 2 rows and 2 columns is the pixel block 240 will be described, but the number of matrices of the set of pixel circuits constituting the pixel block is It is not limited to this. Further, although FIG. 3 describes a case where the pixel blocks 240 are arranged in a 2 × 2 matrix, the number and arrangement of the pixel blocks constituting the pixel array unit 220 are not limited to this.

The pixel block 240 has two reset control signals RST (for example, RST1, RST2), two row selection control signals SEL (for example, SEL1, SEL2), and two transfer control signals (for example, TX1, TX2) in the row direction with respect to the block unit. , The horizontal signal line for supplying the addition signal ADD (for example, ADD1) and the selection control signal ADD_SEL (for example, ADD_SEL1) of the signal after addition is wired.

Further, each pixel circuit 230 is wired with horizontal signal lines for supplying the reset control signal RST, the line selection control signal SEL, and the transfer control signal TX, respectively. Further, horizontal signal lines for supplying the addition signal ADD and the selection control signal ADD_SEL of the added signal are wired for each pixel block. A horizontal signal line for supplying the vertical signal line 410 is wired in the column direction. The output from the pixel is input to the AD conversion circuit 212 at the connection destination via the vertical signal line 410.

By performing the selective driving of each of the horizontal signal lines described above, output is sequentially performed from the image sensor line by line via each vertical signal line 410. As for the output from the pixel array unit 220, when the image pickup apparatus 100 is not operating in the detection mode, the pixel signals of each pixel circuit 230 are sequentially output, whereas in the detection mode, in the pixel block 240 described later. By operating the addition switch for each FD, the addition result is output from a specific pixel.

FIG. 4 is a diagram showing an example of an equivalent circuit of the pixel circuit 230 corresponding to a typical embodiment. FIG. 4 shows an example in which four pixel circuits 230 corresponding to one pixel block 240 are arranged for explanation. Explaining mainly the pixel circuit 230-1 on the upper left, the electric charge generated and accumulated by the photodiode 406-1 is controlled by the transfer control signal TX1 and is floated diffused (hereinafter referred to as FD) 407 via the transfer switch 405. Transfer to -1. The source follower amplifier 408-1 is configured together with a constant current source 411-1 connected to a vertical output line, amplifies a voltage signal based on the electric charge stored in the FD407-1, and outputs it as a pixel signal. The row selection control signal SEL1 controls the row selection switch 409 to connect the output of the source follower amplifier to the vertical output line 410-1.

When resetting the unnecessary charge stored in the FD407-1, the reset switch 404-1 is controlled by the reset control signal RST1. Further, the reset of the photodiode 406-1 is executed by controlling the transfer switch 405-1 by the transfer control signal TX1 together with the reset switch 404-1. The horizontal signal lines that supply the transfer control signal TX1, the reset control signal RST1, and the line selection control signal SEL1 are connected to the vertical scanning circuit 211, and the respective control signals are supplied to each line.

When the image pickup apparatus 100 is not operating in the detection mode, the output signal of each pixel circuit 230 is not added and is read out as a pixel signal from each pixel circuit. In the present embodiment, as a reading method when not in the detection mode, a shooting mode in which a pixel signal is read from each pixel circuit of all pixel blocks and a shooting mode in which a pixel signal is read from each pixel circuit included in some pixel blocks by crop drive are taken. There is a mode. Here, the shooting mode without "crop drive" is called "full screen shooting mode", and the shooting mode with crop drive is called "crop shooting mode". In the crop shooting mode, only the pixel circuit of the pixel block in which the subject change is detected in the immediately preceding detection mode is cropped and shot. The conditions and flow of the shooting mode transition will be described later.

In the crop shooting mode, only a specific pixel circuit selected by the mode control unit 216 is scanned by the vertical scanning circuit 211 and the horizontal scanning circuit 213, and the pixel signal transferred from each pixel circuit 230 is read out. For example, when a subject change is detected in the pixel block 240-1 in FIG. 3 in the detection mode and the mode is changed to the crop shooting mode, the reset control, row selection control, and transfer control in the pixels other than the pixel block 240-1 are performed in the crop shooting mode. , AD conversion is not performed. Therefore, the power for supplying the transfer control signals TX3 and TX4 and the reset control signals RST3 and RST4 and the row selection control signals SEL3 and SEL4 is not consumed. Further, since the AD conversion circuit connected to the vertical output lines 410-3 and 410-4 does not operate, the power for the AD conversion operation is not consumed. Further, since the AD conversion circuit connected to the vertical output lines 410-1 and 410-2 does not perform the AD conversion operation on the pixels of the pixel block 240-3, the corresponding power is not consumed.

In the crop shooting mode, the more pixels that are not read by the crop drive, the less the current consumed by the pixel array unit 220 for reset control, row selection control, and transfer control, and the AD conversion operation of the AD conversion circuit 212. , Power consumption during shooting is reduced.

In the present embodiment, the AD conversion circuit 212 outputs dummy data as the output of the pixel circuit 230 that is not read by crop driving. Therefore, the image size output from the image sensor 200 is the same in the full-screen shooting mode and the crop shooting mode. The value of this dummy data may be set to the same value as the zero output in the pixel, for example.

In the detection mode, the signal transferred to the FD 407 is added and averaged by the FD (407-1, 407-2, 407-3, 407-4) by turning on the switch 413, the switch 414, and the switch 415 with the addition signal ADD. .. After that, the selection control signal ADD_SEL of the added signal is controlled, and the added and averaged signal (added value of the pixel signal) is read out from 230-3 pixels. Since the row selection control signal SEL2 and the selection control signal ADD_SEL of the added signal are input to the row selection switch 409-3 via the OR circuit 412, the source is input when any of the selection control signals is input. The output of the follower amplifier is connected to the vertical output line 410-1. In this way, the added value obtained by adding and averaging the 2 × 2 pixel signals is used as the output of the detection mode. Here, the addition unit of the pixel signal is not limited to 2 × 2. Further, the method of generating the addition value is not limited to the addition in the FD. For example, a pixel circuit having a plurality of rows may be connected to a vertical output line to add and average pixel signals, and horizontal addition may be performed using an adder circuit before the AD conversion circuit.

FIG. 5 is a diagram showing a configuration example of the event detection unit 214 corresponding to a typical embodiment. The output signal of the AD conversion circuit 212 is input to the event detection unit 214. The output switching unit 260 receives the control signal from the mode control unit 216 and switches the output destination of the input signal according to the shooting mode selected as the drive control of the image sensor 200. When the shooting mode selected as the drive control of the image sensor 200 is the detection mode for detecting the subject change, the output switching unit 260 switches the output destination of the pixel signal to the integration calculation processing unit 261. When the shooting mode is not the detection mode, the output switching unit 260 switches the output destination of the pixel signal to the signal processing unit 215.

The integration calculation processing unit 261 integrates the pixel signal values output from the output switching unit 260, and supplies the integrated data to the exposure control unit 217. Further, the data is output to the memory 262 that holds the data in pixel block units. The memory 262 stores and holds the signal (added value of the pixel signal) added and averaged in pixel block units in association with the position information in the plane of the image sensor 200 of the corresponding pixel block.

The difference detection unit 263 acquires a difference value from the latest added value of the pixel signal of each pixel block and the added value of the past pixel signal of the pixel block at the same position held in the memory 262. .. Here, the added value of the pixel signals used for acquiring the difference value is an added value of pixel signals that are temporally back and forth or adjacent to each other in the pixel blocks at the same position. The acquired difference value is supplied to the comparison unit 264. The comparison unit 264 compares the acquired difference value with a predetermined threshold value, and determines whether or not there is an event detection. The result obtained by the comparison unit 264 is transmitted to the control unit 103 via the mode control unit 216. Based on the result, the control unit 103 determines a region in which an event such as a subject change is detected occurs in the divided region in the plane of the image sensor 200.

FIG. 6 is a diagram illustrating an example of detection of captured images and subject changes corresponding to the embodiment. FIG. 6A shows an example in which the screen is divided into 4 × 4 by pixel blocks. The broken line in the figure indicates the area range in pixel block units. For convenience of explanation, the position of the pixel block on the screen is 00 from the upper left of the screen, 01 on the right side of the screen, and 33 on the lower right of the screen. The row is represented by the number on the left side and the column is represented by the number on the right side.

FIG. 6B is an example of a subject on the shooting screen, and shows that the house is located in the lower right 2 × 2 block as the subject. FIG. 6C is a diagram schematically showing the output of each pixel block 240 of the pixel array unit 220 with respect to the subject shown in FIG. 6B. In the figure, the output from the region located in the house portion is obtained from the pixel blocks 22, 23, 32, 33, and the output is the output obtained by adding and averaging the pixel signals from the house and its background. In this example, the case where the background to the house is bright is shown, and the pixel blocks 22 and 23 have lower output values than the pixel blocks 32 and 33 due to the ratio of the house and the background.

FIG. 6D is a diagram showing an example of a subject as in FIG. 6B, and in the shooting situation of FIG. 6B, a person is added to one block at the lower left as a change of the subject. FIG. 6E shows a schematic diagram showing the output of each pixel block for the subject in FIG. 6D. Here, consider a case where the state of the photographing region changes from the state shown in FIG. 6 (B) to the state shown in FIG. 6 (D). The difference detection unit 263 of the event detection unit 214 has the output in pixel block units shown in FIG. 6 (C) held in the memory 262 as past data, and the pixel block shown in FIG. 6 (E) that has been read most recently. The difference value for each pixel block at the same position is acquired from the output of the unit.

The acquired difference value is compared with a predetermined threshold value by the comparison unit 264, and as a result, when the difference value of the pixel block 30 is equal to or more than the predetermined threshold value, the subject change is detected. Since this difference value is associated with position information in the plane of the image sensor (for example, information indicating a two-dimensional matrix division position; in the case of FIG. 6E, information indicating a pixel block 30). , It is possible to determine at which position in the image pickup surface the subject change is detected.

Next, the operation of the image pickup apparatus 100 corresponding to the typical embodiment will be described with reference to FIG. 7. FIG. 7 is a flowchart showing an example of the operation of the image pickup apparatus 100 corresponding to a typical embodiment, and shows an operation flow in the detection mode. The operation will be described below according to the flowchart.

After the image pickup device 100 is activated, the control unit 103 controls the image pickup device 200 to permit the detection mode operation in S701, and shifts to the detection mode. Next, in S702, the control unit 103 controls the image sensor 200 to output a signal (added value of the pixel signal) obtained by adding and averaging the pixel signals for each pixel block, and temporarily stores the signal in the memory 262.

Next, in S703, the control unit 103 determines whether or not the added value of the pixel signals for each pixel block obtained in S702 is the second and subsequent pixels after the start of operation. If it is the second and subsequent sheets, the process proceeds to S704, and if it is less than the second sheet, the process returns to S702. This is because if it is less than the second image, the difference from the past data for detecting the change in the subject cannot be obtained. In S704, the difference detection unit 263 acquires the difference value for each pixel block with respect to the added value of the pixel signal acquired in S702. That is, the difference value between the added value of the pixel signal acquired in S702 held in the memory 262 and the added value of the pixel signals having a temporally front-to-back relationship in the same pixel block is acquired.

Next, in S705, the comparison unit 264 determines the presence / absence of event detection based on the difference value acquired in S704. The comparison unit 264 compares each difference value with a predetermined threshold value, and determines whether or not the difference value is equal to or greater than the threshold value. Then, if there is a difference value equal to or greater than the threshold value, the number of pixel blocks is counted. If there is no count (= 0), the process returns to S702, assuming that there is no change detection event, and the operation of detecting the subject change up to S704 described above is repeated. On the other hand, if the count is counted (> 0), it is assumed that the event has been detected, and the process proceeds to S706.

Next, in S706, the control unit 103 determines whether or not the number of pixel blocks with subject changes counted in S705 is a predetermined number n or more. Here, the number of pixel blocks in which the subject change is detected is determined, which is equivalent to determining the ratio of the pixel blocks in which the subject change is detected. In other words, in S706, it is determined whether or not the ratio of the pixel blocks in which the subject change is detected exceeds a predetermined ratio. If the number of counted pixel blocks is less than n, it is assumed that there is a subject change only in a limited range in the imaging surface, and the process proceeds to S711. Further, in the case of n or more, it is assumed that the subject has changed in a wide range in the imaging surface, and the process proceeds to S707. In this way, the detection mode is temporarily stopped according to the subject detection, and the mode shifts to the full-screen shooting mode or the crop shooting mode.

In S707, the control unit 103 controls the image sensor 200 to permit the operation of the full-screen shooting mode, and shifts to the full-screen shooting mode. Next, in S708, the control unit 103 resets the count value CNT that counts the number of images output in the full-screen shooting mode to 0 after shifting to the full-screen shooting mode. In the subsequent S709, the control unit 103 controls the image sensor 200 to perform shooting in the full-screen shooting mode, outputs image data, and increments the count value CNT. In the following S710, the control unit 103 determines whether or not the count value CNT is equal to or greater than the value N, which is a predetermined number of images (predetermined number of images). If the counter value CNT is less than N, the process returns to S709 and continues the full-screen shooting mode. On the other hand, when the counter value CNT is N or more, the process returns to S701 in order to return to the detection mode again from the full-screen shooting mode and detect the subject change. By the loop from S708 to S710, the detection mode is restarted after shooting a predetermined number of images or more in the full-screen shooting mode or after continuing for a certain period of time.

On the other hand, in S711, the control unit 103 controls the image sensor 200 to permit the operation of the crop shooting mode, and shifts to the crop shooting mode. Next, in S712, the coordinate information in the image pickup surface of the pixel block detected in S705 is transmitted from the event detection unit 214 to the mode control unit 216, and the mode control unit 216 uses the image capture area of the pixel block whose event is detected as the crop area. specify. Next, in S713 and S714, the control unit 103 resets the count values CNT and CNT2, which count the number of images output in the crop shooting mode, to 0, respectively. The count value CNT is a variable representing the number of images taken after the event is detected, and the count value CNT2 is a variable for determining the execution timing of the subject change detection shooting in the crop shooting mode. In the present embodiment, as an example, when M or more image data are output in the crop shooting mode, a case where the subject change is detected while continuing the crop shooting mode will be described.

In S715, the control unit 103 controls the image sensor 200 to perform shooting in the crop shooting mode, outputs image data, and increments the count values CNT and CNT2, respectively. In S716, the control unit 103 determines whether or not the count value CNT2 is equal to or greater than a predetermined value M. When the counter value CNT2 is less than M, the process returns to S715, repeats after S715, and continues the crop shooting mode. On the other hand, when the counter value CNT2 is M or more, the process proceeds to S717 in order to periodically detect the subject change in the crop shooting mode.

In S717, the control unit 103 controls the image sensor 200 to perform shooting by driving for detection. Similar to S702, the added value of the pixel signal is output for each pixel block in the image sensor 200, and is temporarily stored in the memory 262. At this time, shooting is performed in the entire imaging region without cropping, and the added value of the pixel signals is output. Next, in S718, the difference detection unit 263 acquires the difference value between the added value of the pixel signal acquired in S717 and the added value of the past pixel signal held in the memory 262 for each pixel block. Next, in S719, the comparison unit 264 determines the presence / absence of event detection based on the difference value obtained in S718. The details of the determination here are the same as those described in S705. As a result of the determination, if there is no change detection event, the process proceeds to S720. If there is an event detection, the process proceeds to S721.

In S720, the control unit 103 determines whether or not the count value CNT is equal to or greater than a predetermined value N. If the counter value CNT is less than N, the process returns to S714 and continues the crop shooting mode. When the counter value CNT is N or more, the process returns to S701 in order to shift from the crop shooting mode to the detection mode. By the loop from S713 to S720, the operation of restarting the detection mode after continuing the crop shooting mode for a certain period of time is performed.

On the other hand, in S721, the comparison unit 264 determines whether or not the number of pixel blocks with subject changes counted in S719 is a predetermined number n or more. If the number of counted pixel blocks is less than n, it is assumed that there is a subject change only in a limited range in the imaging surface, and the process proceeds to S712. Further, in the case of n or more, it is assumed that the subject has changed in a wide range in the imaging surface, and the process proceeds to S707.

In this way, in an imaging device that detects changes in the subject as an event, by cropping only the imaging area where the event is detected and shooting, it is possible to reduce power consumption even when the event is frequently detected. An imaging device can be provided. Further, in the detection mode, the power consumption in the detection mode can be reduced by performing the event detection of the subject change in the drive mode in which the pixels are added and output for each pixel block. In addition, when the subject change is detected in the detection mode and the area where the event is detected is a part of the imaging surface, the power consumption at the time of shooting can be increased by photographing only the imaging surface area where the subject changes in the crop shooting mode. The data size of the stored image can be reduced.

Also, in the crop shooting mode, shooting is performed at regular intervals in the detection mode for subject detection, and the cropping area is changed to the subject change by reacquiring the imaging surface area where the subject changes and updating the crop area. It can be made to follow. Therefore, even when an event is frequently detected and the mode is frequently changed to the shooting mode, the movement of the subject to be shot can be recorded while reducing the power consumption at the time of shooting and the data size of the stored image. Can be done.

Next, the structure of the image pickup device 200 will be described with reference to FIG. In the above embodiment, the image sensor 200 can be configured to have a semiconductor substrate 801 shown by diagonal lines as shown in FIG. 8A and a semiconductor substrate 802 shown in white. The semiconductor substrate 801 and the semiconductor substrate 802 are sealed and modularized (integrated) in a superposed state as shown in FIG. 8B. As a result, as shown in FIG. 8C, the semiconductor substrate 801 and the semiconductor substrate 802 form a multilayer structure (laminated structure). The circuit formed on the semiconductor substrate 801 and the circuit formed on the semiconductor substrate 802 are connected to each other by vias (VIA) or the like. Although the case of two layers is shown in FIG. 8, the number of layers may be increased.

As described above, the image sensor 200 may be formed as a module (also referred to as an LSI (Large Scale Integration) chip) in which the semiconductor substrate 801 and the semiconductor substrate 802 are integrated so as to form a multilayer structure. By forming the semiconductor substrate 801 and the semiconductor substrate 802 in a multilayer structure in this way inside the module, the image sensor 200 can realize a larger-scale circuit mounting without increasing the size of the semiconductor substrate. .. That is, the image sensor 200 can mount a larger-scale circuit while suppressing an increase in cost.

A pixel array unit 220, an AD conversion circuit 212, a vertical scanning circuit 211, a horizontal scanning circuit 213, and the like are formed on the semiconductor substrate 801. Further, the event detection unit 214, the signal processing unit 215, the mode control unit 216, and the exposure control unit 217 can be formed on the semiconductor substrate 802.

Further, in the above-described embodiment, the detection mode has been described as a shooting mode in which the number of output pixels is reduced by adding and reading a plurality of pixel signals in a predetermined block, but the method for reducing the number of pixels is limited to this. It's not something. For example, the number of output pixels may be reduced by periodically thinning out the pixels to be read. At that time, the pixels in the pixel block may be thinned out at predetermined intervals, and then the pixel signals may be added to detect the change in the subject.

Further, when a color filter or the like is arranged on the image sensor to output pixels for each color, pixel signals of the same color may be added according to rows or columns in which pixels of the same color are arranged. At that time, the pixel signals of only the pixels of a part of the plurality of colors may be added. For example, in the case of RGB, a pixel signal of only G pixels may be added.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

100: Image pickup device, 200: Image pickup device

Claims (11)

An image pickup device in which pixel circuits are arranged in a matrix and includes an image pickup element capable of outputting an addition value of pixel signals for each block composed of a plurality of pixel circuits and a pixel signal value of each pixel circuit.
A detection means for detecting a subject change based on the difference between the added values of the pixel signals output from the block at the same position in the image sensor and which are back and forth in time is provided.
When the detection means detects a subject change, the image sensor outputs the pixel signal value of each pixel circuit included in the block for the block in which the subject change is detected, and the block in which the subject change is not detected is output. The image pickup device is characterized in that it does not output the pixel signal value of each pixel circuit included in the block. When the detection means detects a subject change in a block exceeding a predetermined ratio among all the blocks in the image sensor, the image sensor detects pixels of each pixel circuit included in each of the blocks. The image pickup apparatus according to claim 1, wherein a signal value is output. The image sensor outputs the pixel signal value of each pixel circuit included in each of the blocks to resume the output of the added value of the pixel signal for each block after taking a predetermined number of images. 2. The image pickup apparatus according to claim 2. The first or second aspect of the present invention, wherein the detection means stops the detection of the subject change when the output of the pixel signal value of each pixel circuit is started in response to the detection of the subject change. Imaging device. After stopping the detection of the subject change, the detection means outputs the pixel signal value of each pixel circuit to restart the detection of the subject change after taking a first predetermined number of images or more. The imaging apparatus according to claim 4. After the detection means stops detecting the change in the subject, the image sensor outputs the added value of the pixel signal for each of the blocks after taking the first predetermined number of images or more.
The detection means further detects the subject change based on the difference between the added values of the pixel signals output from the block at the same position and before and after the time.
When the detection means further detects the subject change, the image sensor outputs the pixel signal value of each pixel circuit included in the block for the block in which the subject change is further detected, and the subject change is detected. For the block that was not executed, the pixel signal value of the pixel circuit included in the block is not output.
When the detection means does not further detect the change in the subject, the image sensor outputs the pixel signal value of each pixel circuit until a second predetermined number of images larger than the first predetermined number of images are captured. The image pickup apparatus according to claim 5. The added value of the pixel signal is acquired by the summing average of all the pixel signals of the plurality of pixel circuits included in the block, or the summing average of the pixel signals of at least a part of the pixel circuits of the plurality of pixel circuits. The imaging device according to any one of claims 1 to 6, wherein the image pickup apparatus is characterized by the above. The image pickup device according to any one of claims 1 to 7, wherein the image pickup device outputs dummy data as a pixel signal value of a pixel circuit included in a block in which a change in the subject is not detected. The image pickup device according to any one of claims 1 to 8, wherein the image pickup device includes the detection means. The image pickup device has a structure in which a plurality of substrates are laminated, and has a structure in which a plurality of substrates are laminated.
The image pickup apparatus according to claim 9, wherein the pixel circuit and the detection means are formed on a different substrate. It is a control method of an image pickup device including an image pickup element in which pixel circuits are arranged in a matrix and can output an addition value of pixel signals for each block composed of a plurality of pixel circuits and a pixel signal value of each pixel circuit. ,
A detection step in which the detection means detects a subject change based on the difference in the added values of the pixel signals output from the block at the same position in the image sensor before and after the time.
When a subject change is detected in the detection step, the block including the subject change detected block includes an output step of outputting a pixel signal value of each pixel circuit included in the block.
A control method for an imaging device, characterized in that, in the output step, the pixel signal value of each pixel circuit included in the block is not output for the block in which the subject change is not detected.

Images