JP7408294B2 - Imaging device, lens device, method of controlling the imaging device, and method of controlling the lens device - Google Patents

Description

The present invention relates to an imaging device, a lens device, a method of controlling an imaging device, and a method of controlling a lens device.

There is an imaging device that includes a shake correction member in both an interchangeable lens and a camera body. It is expected that by controlling the blur correction functions of both systems in a coordinated manner, it will be possible to effectively utilize the respective blur correction ranges. By controlling each correction member using the gyro sensor information obtained by both the interchangeable lens and the camera body, it is possible to obtain a photographed image in which blur is favorably corrected. Patent Document 1 calculates the difference between the residual blurring correction calculated using a motion vector detected between captured images and camera attitude information acquired from a gyro sensor, and identifies the offset of the gyro sensor information. A technique for correcting gyro sensor information is disclosed.

JP2017-92616A

However, in Patent Document 1, when two gyro sensors are used, one on the interchangeable lens side and the other on the camera body side, the offset of each gyro sensor cannot be identified. In the case of cooperative image stabilization using two image stabilization members and two gyro sensors, the offset component of the two gyro sensors is added to the remaining amount of blur correction calculated using the motion vector detected from the image. This is because they are superimposed. In this way, in image stabilization using multiple image stabilization members and multiple image blur detection units, the amount of error from the multiple image stabilization members is superimposed on the remaining amount of blur correction calculated from the image. The accuracy of estimating the error amount of the shake correction member becomes lower.

An object of the present invention is to provide an imaging device that improves the accuracy of image blur correction in an imaging device that has a plurality of shake detection units and blur correction units.

In order to solve the above problems, an imaging device of the present invention is an imaging device with a detachable lens device, and includes a vector detection section that detects a motion vector from a plurality of images, and a vector detection section that detects shake information of the imaging device. A first shake correction amount is obtained based on the first shake information detected by the first shake detection section and the first shake detection section, and an image is adjusted based on the obtained first shake correction amount. a first shake correction unit that corrects the shake of the camera; an error estimation unit that estimates an error included in the first shake information detected by the first shake detection unit ; and a communication means for receiving a second blur correction amount that is a blur correction amount of a second blur correction unit . The error estimator is configured to calculate the first shake detected by the first shake detector based on the motion vector detected by the vector detector and the second shake correction amount received by the communication means. The first blur correction unit estimates an error included in the information , and obtains the first blur correction amount based on the estimation result by the error estimation unit.

According to the present invention, it is possible to provide an imaging device that improves the accuracy of image blur correction in an imaging device that has a plurality of shake detection units and shake correction units.

1 is a diagram showing the configuration of an imaging device. FIG. 3 is a diagram showing the configuration of first and second blur correction sections. 3 is a flowchart showing blur correction processing. 3 is a flowchart showing motion vector detection processing. FIG. 2 is a diagram illustrating an overview of template matching. It is a flow chart which shows the 1st shake correction processing. FIG. 3 is a diagram illustrating calculation of a blur correction amount. It is a flowchart which shows error estimation processing. It is a flowchart which shows error estimation processing.

(Example 1)
FIG. 1 is a diagram showing the configuration of an imaging system 100. The imaging system 100 is, for example, an imaging device main body (digital camera) with interchangeable lenses and a lens device attached to the digital camera. The imaging system 100 of this embodiment can correct image blur using two different means. The imaging device that constitutes the imaging system 100 includes a camera system control unit (hereinafter referred to as a control unit) 101, a memory 102, an imaging unit 104, a development processing unit 105, and a motion vector detection unit (hereinafter referred to as a vector detection unit) 106. Equipped with. Furthermore, the imaging system 100 includes a first blur correction section 107 and a recording medium 110. Further, a lens device attached to the imaging device main body constituting the imaging system 100 includes an optical system 103 and a second blur correction section 108. Further, each of the imaging device main body and the lens device is provided with a communication means for transmitting the image stabilization information, and the communication means of the imaging device main body and the communication means of the lens device are each equipped with a communication means for transmitting the image stabilization information. A communication unit (hereinafter referred to as communication unit) 109 is configured.
In this embodiment, an example will be described in which image blur is corrected by two different means, but the number of means for correcting image blur is not limited to two as long as it is plural.

The control unit 101 is, for example, a CPU (Central Processing Unit), and controls the entire imaging system 100. The control unit 101 controls the operation of each block by reading out the operation program for each block of the imaging system 100 from the memory 102, expanding it, and executing it. The memory 102 includes a ROM (Read Only Memory) and a RAM (Random Access Memory). The ROM is a rewritable nonvolatile memory, and stores operation programs for each block included in the imaging system 100 as well as parameters necessary for the operation of each block. The RAM is a rewritable volatile memory, and is used as a temporary storage area for data output during the operation of each block included in the imaging system 100. Note that the control unit 101 may be included in the lens device, or may function as the control unit 101 described below by cooperation between the CPU of the imaging device main body and the CPU of the lens device.

The optical system 103 is an imaging optical system composed of lenses and the like, and includes a focus lens, a zoom lens, a shift lens, and their drive systems. The imaging unit 104 includes an imaging element that photoelectrically converts the subject image formed by the optical system 103. Examples of the image sensor include a CCD (Charge Coupled Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) image sensor. In this embodiment, the image sensor is capable of wobbling, and by wobbling in a direction that cancels out the shake occurring in the image pickup device, blur correction can be performed. The development processing unit 105 forms an image signal from the electrical signal output from the imaging unit 104. The development processing section 105 includes an A/D conversion section, an auto gain control section (AGC), and an auto white balance section, and forms a digital signal from an electric signal. The imaging unit 104 and the development processing unit 105 constitute an imaging system that acquires images.

The vector detection unit 106 uses the image input from the development processing unit 106 and the image stored in the memory 102 to detect the amount of movement of an arbitrary pixel or area between a plurality of images as a motion vector. The first blur correction unit 107 and the second blur correction unit 108 correct image blur. Therefore, the imaging system 100 can perform blur correction using two different blur correction units. A detailed explanation of the first blur correction section 107 and the second blur correction section 108 will be given later using FIG. 2.

The communication unit 109 communicates the movement of the shake and the like between the first shake correction unit 107 included in the imaging device main body and the second shake correction unit 108 included in the lens device as shake correction information necessary for estimating the amount of error. Sends and receives information to correct blur. The recording medium 110 is a storage device or storage medium for storing and holding images subjected to blur correction processing, parameters at the time of shooting, and the like.

In this embodiment, the shake of the imaging system 100 refers to temporal fluctuations in the position and orientation of the imaging system 100, and includes intentional camera work by the photographer, camera shake movements, etc. It is. Note that the term "image blur" described in this embodiment does not mean to suppress physical vibrations such as shake or shake, but rather to correct image blur in an image due to shake or shake. means.

FIG. 2 is a diagram showing the configuration of the first blur correction section 107 and the second blur correction section 108. As described above, the first blur correction section 107 is provided in the imaging device main body, and the second blur correction section 108 is provided in the lens device. The first blur correction section 107 includes a first blur detection section 201 , a first blur correction control section 202 , a first correction member 203 , and a first error estimation section 207 . The first shake detection unit 201 detects shake occurring in the imaging system 100. The first blur correction control unit 202 uses the information about the shake detected by the first blur detection unit 201 to calculate a blur correction amount for correcting image blur. Then, the first blur correction control unit 202 uses the calculated blur correction amount to swing control the first correction member 203 so as to suppress image blur.

The first error estimation unit 207 calculates the true value of the blur from the remaining amount of blur obtained from the motion vector detected by the vector detection unit 106 and the amount of blur correction generated by filtering the detected amount of blur. The amount of error in blur correction is estimated by calculating the difference between Information is exchanged between the first blur correction section 107 and the second blur correction section 108 when performing the blur correction process via the communication section 109.

The second shake correction section 108 includes a second shake detection section 204 , a second shake correction control section 205 , a second correction member 206 , and a second error estimation section 208 . Each part of the second blur correction section 108 plays the same role as each part of the first blur correction section 107. The first shake detection section 201 and the second shake detection section 204 may have the same configuration or may have different configurations. For example, while the first shake detection section 201 uses a gyro sensor, the second shake correction section 205 may use a gyro sensor in the same way as the first shake detection section 201; A sensor may be used to detect movement of the imaging system 100. Note that both the first shake detection section 201 and the second shake detection section 204 detect movement of the imaging system 100. However, in order to distinguish between the two shake detection sections, the first shake detection section provided in the imaging device body is replaced with the detection section that detects shake of the imaging device, and the second blur detection section provided in the lens device. The part is sometimes called a detection part that detects shake of the lens device.

The first correction member 203 and the second correction member 206 are members that are subject to rocking control for image blur correction, and are different members. In this embodiment, as an example, the first correction member 203 is a movable image sensor mounted in the main body of the imaging device 100, and the second correction member 206 is a shaker mounted on the lens of the optical system 103. A correction optical system (for example, a shift lens that is a correction lens) is used. However, these examples are merely examples, and there is no particular restriction as long as the members to be subjected to rocking control for blur correction are not the same. Furthermore, the method of image blur correction is not limited to optical correction by controlling the rocking of members; the blur correction performed by the first blur correction unit 107 involves electronically performing geometric deformation processing on the image. Image processing may also be used.

FIG. 3 is a flowchart showing blur correction processing. In step S301, the vector detection unit 106 detects a motion vector between the two input frame images. Vector detection section 106 outputs the detected motion vector to first error estimation section 207. Details of step S301 will be described later using FIG. 4.

In step S302, the first blur correction unit 107 performs blur correction processing (first blur correction processing) to correct blur. Specifically, the first blur correction unit 107 uses the amount of motion (motion vector) between frame images detected in S301 and the correction amount used in the blur correction processing in S302 to perform the second blur correction process. Information for correcting the shake information of the imaging system detected by the shake correction unit 108 is generated. Details of step S302 will be described later using FIG. 6. In step S303, the second blur correction unit 108 performs blur correction processing (second blur correction processing) to correct blur.

FIG. 4 is a flowchart showing the motion vector detection process performed in step S301. In step S401, the imaging unit 104 and the developing unit 105 acquire an image. Specifically, first, the imaging unit 104 performs photoelectric conversion and outputs the subject image formed by the optical system 103 as an analog signal corresponding to the subject brightness. The development processing unit 105 that has acquired the analog signal from the imaging unit 104 converts the analog signal into a digital signal using an A/D conversion unit. Further, the development processing unit 105 performs signal level correction and white level correction on the digital signal using AGC and AWB to generate an image signal. The generated image is output to the vector detection unit 106.

In step S402, the vector detection unit 106 detects motion vectors between the plurality of input images. In this embodiment, the vector detection unit 106 detects a motion vector from two images. In this embodiment, a method using template matching will be described as an example of a motion vector detection method.

FIG. 5 is a diagram illustrating an overview of template matching. In template matching, motion vectors are detected using an original image and a reference image as multiple images. 5(A) is an original image, and FIG. 5(B) is a reference image, and these images are images photographed through the optical system 103 or images stored and held in the memory 102. The template block 501 is a template block placed at an arbitrary position in the original image.

In template matching, a correlation value between the template block 501 and each region of the reference image is calculated. At this time, calculating the correlation value for the entire area of the reference image requires a huge amount of calculation, so the search range is set as a rectangular area for calculating the correlation value on the reference image, as shown in the search range 502. Set. There are no particular restrictions on the position or size of the search range 502. The correlation value target block 503 is an area within the search range 502 that is the target for calculating the correlation value with the template block 501, and has the same size as the template block 501.

In this embodiment, a sum of absolute differences (hereinafter abbreviated as SAD) is used as an example of a method for calculating a correlation value. The formula for calculating SAD is shown in Formula 1.

In Equation 1, f(i,j) represents the brightness value at coordinates (i,j) within template block 501. g(i,j) represents each brightness value within the correlation value target block 503 that is the target of correlation value calculation in the search range 502. In SAD, the correlation value S_SAD can be obtained by calculating the absolute value of the difference for each luminance value f (i, j) and g (i, j) in both blocks and calculating the sum thereof. Therefore, the smaller the value of the correlation value S_SAD, the smaller the difference in luminance value between the template block 501 and the correlation value target block 503, that is, the more similar the textures of the template block 501 and the correlation value target block 503 are. There is. In this embodiment, SAD is used as an example of the correlation value, but the present invention is not limited to this, and other correlation values such as sum of squared differences (SSD) and normalized cross-correlation (NCC) may be used. Also good.

The correlation value target block 503 is moved for the entire area of the search range 502, and the correlation value with the template block 501 is calculated. Then, by determining the position where the correlation value between the template block 501 and the search range 502 is the smallest, it is possible to detect to which position the template block on the original image has moved in the reference image, and to detect the movement between images. It becomes possible to detect vectors. The motion vector detection process as described above is performed in a plurality of regions between input frame images. The detected motion vector is then transmitted to the first error estimator 207. Note that in this embodiment, a method using template matching has been described as an example of vector detection, but other methods such as a method using a gradient method and corresponding point search using feature point extraction may also be used.

Next, blur correction processing will be explained. First, the first blur correction process in step S302 in FIG. 3 will be described. FIG. 6 is a flowchart showing the first blur correction process performed in step S302. In the first blur correction process, the first blur correction unit 107 corrects the blur of the imaging device by rocking the first correction member 203. In step S601, the first shake detection unit 201 acquires shake information of the imaging device 100. The first shake detection unit 201 is, for example, a gyro sensor, and acquires shake information in the pan, tilt, and roll directions of the imaging device 100 as shake information. In this embodiment, a gyro sensor is used as a means for acquiring shake information, but the invention is not limited to this, and other means capable of acquiring shake information of the imaging device 100, such as an acceleration sensor or a magnetic sensor, may be used. It's okay. The first shake detection unit 201 then transmits the acquired shake information of the imaging device to the first shake correction control unit 202.

In step S<b>602 , the first shake correction control unit 202 performs shake correction for controlling the first correction member 203 to swing based on the shake information of the imaging system 100 so as to cancel the shake occurring in the imaging system 100 . amount (shake correction amount information). Specifically, the first blur correction control unit 202 performs signal processing on the shake information of the imaging system 100 acquired in step S601, and calculates the amount of blur correction. FIG. 7 is a diagram illustrating calculation of the amount of blur correction in the first blur correction control unit 202.

The first blur correction control unit 202 includes an HPF (high pass filter) 701, an error amount subtraction unit 702, an integration unit 703, a camera work determination unit 704, and an edge processing determination unit 705. The HPF 701 cuts off low frequency components below a set low cutoff frequency among the frequency components of shake information of the imaging system 100 acquired from the first shake detection unit 201, such as angular velocity information, and performs a low cutoff. Outputs high frequency components exceeding the frequency. The low cutoff frequency here is determined by the camerawork determination unit 704.

The error amount subtraction unit 702 performs a process of subtracting a predetermined error amount from the shake information of the imaging system 100 output from the HPF 701. The error amount is estimated by the error estimation unit 207, the details of which will be described later. The integrating section 703 integrates the shake information of the imaging system 100 after the error amount subtraction outputted from the error amount subtracting section 702, and uses the integrated shake information of the imaging system 100 as a shake correction amount. Then, the first blur correction control unit 202 controls the first correction member 203 according to the amount of blur correction. The time constant of integration in the integrating section 703 is determined by the camera work determining section 704.

The camera work determination unit 704 determines whether camera work is included in the shake of the imaging system 100 based on the shake information output from the HPF 701 or the first shake detection unit 201 and the shake information output from the integrating unit 703. Make a judgment. Camera work includes panning, tilting, and other actions added to the imaging system 100 that are intended by the photographer. Further, the camerawork determination unit 704 changes and determines the cutoff frequency of the HPF 701 and changes and determines the integration time constant of the integration unit 703 as camerawork control.

Edge processing determining section 705 determines whether edge processing is necessary based on the amount of blur correction output from integrating section 703. Then, the edge processing determining unit 705 controls, for example, changing the error amount for the error amount subtracting unit 702, depending on the necessity of edge processing. Here, edge processing is when the amount of blur correction reaches the physical movable limit of the correction member or the electrical limit of correction, resulting in an unnatural change in the effect state of the blur correction momentarily. This is a process that prevents images from being generated. As an example of edge processing, processing is performed to gradually lower the vibration suppression rate as the edge approaches. The blur correction amount determined as described above is transmitted to the first blur correction member 203.

In step S603, the first shake correction control unit 202 corrects the shake occurring in the imaging system 100 by controlling the first shake correction member 203 based on the shake correction amount calculated in S602. When the first shake correction member 203 is a correction lens included in the optical system 103, the first shake correction member 203 controls the shift of the correction lens in a plane perpendicular to the optical axis.

移動量ｘが撮像システム１００で制御可能な限界を超える場合には、第１のブレ補正制御部２０２は、補正可能範囲に補正量をクリッピングする。 Note that when the first blur correction member 203 is an image sensor of the imaging unit 102, the first blur correction control unit 202 controls the shift of the image sensor in a plane perpendicular to the optical axis. Further, when the first blur correction member 203 is an image processing circuit that performs electronic correction by cutting out an image, the first blur correction control unit 202 controls the readout range of the image. When the information for controlling the first shake correction member 203 is angle information and the shake correction amount in the first shake correction control unit 202 is angle information, the angle information is used as the control information. It can be used as On the other hand, when the information for controlling the first shake correction member 203 is the amount of parallel movement, and when the amount of shake correction in the first shake correction control unit 202 is angle information, the first The blur correction control unit 202 calculates the amount of parallel movement based on the angle information and uses it as control information. For example, if the position/posture displacement information of the imaging system 100 is the angle information θ and the focal length is f, then the amount of movement x, which is the shake correction amount, is expressed by the following equation 2.

If the movement amount x exceeds the limit that can be controlled by the imaging system 100, the first blur correction control unit 202 clips the correction amount to the correctable range.

In step S604, the error estimation unit 207 estimates the amount of error included in the shake information of the imaging system 100 detected by the first shake detection unit 201. In a method for estimating the amount of error in a conventional imaging device equipped with only one blur correction member, first, the amount of blur correction remaining is calculated using a motion vector detected between acquired frame images after blur correction. Next, the difference between the shake correction amount obtained by the process described in the shake correction control unit 202 and the remaining shake correction is calculated for the shake information of the imaging device acquired by means such as a gyro sensor. This makes it possible to identify the amount of offset included as an error amount in the detected value of the gyro sensor, and adjusts the gyro sensor by using the offset value.

However, when blur correction is performed in the imaging system 100 having two blur detection means and two blur correction members as in this embodiment, the remaining amount of blur correction calculated from the motion vector between images includes two blur correction members. The error amounts generated from each detection means are superimposed. Therefore, with the conventional method, it is not possible to separate the amount of error occurring from each of the two shake detection means, and the shake detection means cannot be adjusted. Therefore, in this embodiment, the two blur correction sections cooperate to estimate the amount of error.

For the sake of explanation, in the following, it is assumed that among the blur correction processing performed by the first blur correction section 107 and the second blur correction section 108, the priority is given to the blur correction processing performed by the first blur correction section 107. The blur correction process in the second blur correction unit 108 will be described as having a relatively low priority. Further, the first shake detection unit 201 of the first shake correction unit 108 is a high-performance gyro sensor, and the second shake detection unit 204 of the second shake correction unit 109 is a gyro sensor with relatively low performance. shall be. Note that the priorities of the blur correction processing between the first blur correction section 107 and the second blur correction section 108 may be exchanged.

As an overview of the estimation of the amount of error by the cooperation of the two shake correction units, first, the amount of shake of the imaging system 100 calculated by the first shake correction unit 107 is sent to the second shake correction unit 108 as the true value of shake. do. When transmitting from the first blur correction section 107, the communication section 109 or the first error estimation section 207 calculates a phase shift due to a delay caused by processing such as communication with respect to the true value of the blur. Make compensation. The second blur correction section 108 uses the true value of the blur received from the first blur correction section 107 to estimate the amount of error of the blur detection means of the second blur correction section 108 . Then, the first shake correction unit 107 and the second shake correction unit 108 estimate the amount of error that minimizes the difference between the true value of the shake and the shake information of the imaging system 100 detected from each shake detection means. This is performed for each shake detection means.

8 and 9 are flowcharts showing the error estimation process in S604. FIG. 8 shows error estimation processing in the first blur correction section 107. FIG. 9 shows error estimation processing in the second blur correction section 108.

In step S801, the first error estimation unit 207 obtains the true value of blur. The first error estimation unit 207 calculates the blurring calculated using the amount of blurring correction calculated by the first blurring correction control unit 202 and the motion vector detected from between the frame images after blurring correction in the motion vector detecting unit 106. Calculate the sum with the remaining amount of correction. This sum value is the movement of the blur of the imaging system 100 detected as the object to be corrected, and in other words, it can be said to be the true value of the blur. The true value of the blur calculated in step S801 is transmitted to the second error estimation unit 208 of the second blur correction unit 108, which has a low priority, after being subjected to phase compensation.

In step S802, the first error estimation unit 207 acquires the observed value of shake information of the imaging system 100. The observed value of shake information is shake information of the imaging system 100 that is directly acquired from the first shake detection unit 201 that is the target of error estimation. In step S803, the first error estimation unit 207 performs error estimation processing using a Kalman filter using the true value of the blur of the imaging system calculated in step S801 and the observed value of the amount of blur obtained in step S802.

In step S804, the first error estimation unit 207 uses the error amount updated by the Kalman filter estimation process for the error amount subtraction process in the blur correction control unit 201 of the first blur correction unit 107. In step S805, the first error estimation unit 207 uses the estimated error value to determine whether to update the priorities of the first blur correction unit 107 and the second blur correction unit 108, and in subsequent processing. It is determined which blur correction unit's error estimation processing should be prioritized.

As a method for determining whether to update the priority, for example, there is a method using the deviation of the error amount calculated in the error amount estimation process in step S803. Observe the temporal fluctuations in the deviation of the error amount, and if the value is close to the true value, it means that the error amount has been calculated and adjusted accurately and the motion information of the imaging system 100 can be stably detected. Therefore, the other image stabilization unit is given a higher priority. Further, the priority may be given to a blur correction unit with a large error calculated in the error amount estimation process.

Furthermore, the priority may be set depending on the detection accuracy of the shake detection section. For example, the priority may be given to a blur correction section that has a blur detection section with high detection accuracy, or the priority may be given to a blur correction section that has a blur detection section that has low detection accuracy at low frequencies. Furthermore, the priority may be set depending on the correction performance of the blur correction section. For example, a high priority is set for a blur correction unit that can correct blur at lower frequencies. Alternatively, the error of each blur correction section may be estimated alternately.

In step S806, the first error estimating unit 207 switches the blur detection unit to which the error is to be corrected to the blur detection unit of the other blur correction unit in accordance with the priority changed in step S805. By continuing to repeat the switching of the correction target as described above, it becomes possible to stably and accurately acquire shake information of the imaging system 100 from either shake detection unit.

In this embodiment, the method of updating the priority of the blur correction unit that performs error estimation according to the deviation of the error amount has been described, but the method of updating the priority is not limited to this. Another method is to update the priority when the priority is below. Furthermore, there is also a method of automatically switching the priority every time a predetermined period passes without performing a priority update determination.

In step S605 in FIG. 6, the first error estimation unit 207 or the communication unit 109 performs phase compensation processing on the true value of the blur calculated in step S801 in FIG. When transmitting the true value of the blur calculated in the first error estimation section 207 to the second error estimation section 208, the A phase difference occurs between the amount of blur correction calculated by the blur correction control unit 205 of No. 2. Therefore, by performing phase compensation on the true value of blur calculated by the first error estimation unit 207, the phase shift between the two is compensated for. There are no particular restrictions on the phase compensation method, and general techniques such as lowering the gain in high bands may be used. The true value of the phase-compensated blur is then transmitted to the second error estimation section 208 via the communication section 109.

Next, the second blur correction process in step S303 in FIG. 3 will be explained. Here, first, using the shake information of the imaging system 100 obtained from the second shake detection unit 204 that constitutes the second shake correction unit 108, the second shake correction control unit 205 calculates the amount of shake correction. . Next, the vibration of the imaging system 100 is corrected by drive-controlling the second vibration reduction member 206 based on the calculated vibration correction amount. Then, the second error estimation unit 208 calculates the amount of error included in the shake information of the imaging system 100 detected by the second shake detection means, and adjusts the shake detection means using the calculated value. Here, the content of the processing in each component is the same as that of the first shake correction unit 108 described above, but the processing may be performed using processing parameters that match the type of shake detection means and shake correction member. become.

Here, error estimation processing by the second error estimation unit 208 will be explained. FIG. 9 shows error estimation processing in the second blur correction section 108. In step S901, the second error estimation unit 208 obtains the true value of blur calculated in processing step S801 of the first error estimation unit 207 as the true value of blur. This is because the blur detection accuracy of the first blur detection section 201 is higher than the blur detection accuracy of the second blur detection section 204 in this embodiment. The second error estimation unit 208 calculates the remaining blur correction amount using the blur correction amount calculated from the second blur correction control unit 205 and the motion vector detected from between frame images in the motion vector detection circuit 106. It is also possible to obtain the true value of blur by calculating the sum of However, when a gyro sensor with low performance is used, the influence of errors and noise components of the gyro sensor is large, and it may not be possible to accurately determine the true value of blur. Even if a subsequent error estimation process is performed using the true value of blur that includes many such errors, an incorrect error value will be calculated, making it impossible to accurately adjust the gyro sensor. Therefore, in this embodiment, by using the true value of the shake calculated by the first shake correction unit 107, which is calculated using a high-performance gyro sensor, when a low-performance gyro sensor is used, However, the gyro sensor should be able to be adjusted accurately.

The first image stabilization unit 107 uses a high-performance gyro sensor, whereas the second image stabilization unit 108 uses a relatively low-performance gyro sensor. It is not limited to. For example, the configuration may be such that the first blur correction section 107 corrects low frequency blur, and the second blur correction section 108 corrects high frequency blur. In this case, the blur correction section that corrects low-frequency blur is given a relatively high priority, and the blur correction section that corrects high-frequency blur is given a relatively low priority. This is because offset components are more likely to be included as errors when low-frequency vibrations are detected, and correcting this offset component improves image stabilization performance more than errors occurring on the high-frequency side. This is because the influence of In addition, if there is a difference in the estimation performance of the amount of blur correction or error amount between the first image stabilization unit 107 and the second image stabilization unit 108, a relative priority is set according to the estimation performance. You can. Further, the true value of the blur acquired here may be based on the shake information detected by the shake detection section included in the correction section with the higher priority. Therefore, instead of acquiring the true value of the shake acquired by the first shake correction unit 107, the shake information itself detected by the first shake detection unit 201 or the shake correction acquired based on the shake information is used. The second error estimation unit 208 may obtain the true value of the blur by obtaining the amount and motion vector information.

Steps S902 to S904 are the same as steps S802 to S804 in FIG. 8, so their explanation will be omitted. However, the observed value acquired in step S902 is shake information of the imaging system 100 detected by the second shake detection unit 204, and the error value acquired in step S904 is the shake information acquired by the second shake correction unit 108. This is the error value included in the blur correction amount. The subsequent processes (steps S805 and S806) are performed by the first error estimating unit 209, so that the second blurring correction unit 108 is not required. By estimating the error included in the amount of shake correction obtained by the first shake correction unit 107 and the second shake correction unit 108 as described above, and performing shake correction using the estimated error value, The blur correction system can be improved. Further, the blur corrected image generated as described above is stored in the storage medium 110.

As explained above, according to this embodiment, in an imaging apparatus having a plurality of shake detection means and shake correction members, each shake correction control unit cooperates to estimate the amount of error, so that the amount of error included in shake information is It is possible to adjust the error amount with high precision. This makes it possible to calculate the amount of blur correction with high precision, and improve image blur correction performance. Note that in this embodiment, since the imaging apparatus main body includes the first shake correction unit 107 which has a high priority, the motion vector detection unit 106 transmits information on the remaining amount of shake correction to the first error estimation unit 207. did. However, if the second blur correction unit 108 included in the lens device has a higher priority, the motion vector detection unit 106 transmits information on the remaining amount of blur correction to the second error estimation unit 208. In this case, since communication is performed between the lens device and the imaging device main body, it is preferable to perform phase compensation processing also on information on the remaining amount of blur correction. As with the process for the true value of blur, there are no particular restrictions on the phase compensation process, and conventional processes can be used. The true value of the blur can be obtained by the sum of the remaining blur correction amount after the phase compensation process and the blur correction amount obtained by the second blur correction control unit 205.
Furthermore, in the present embodiment, a configuration has been described in which each of the first blur correction section 107 and the second blur correction section 108 includes an error estimation section, but even if only one of them includes an error estimation section. good. The true value of the shake is acquired based on the shake information and the motion vector detected by one of the shake detection units, and the true value of the shake and the shake information detected by the detection unit included in the correction unit that is the error target are estimated. By using , it is possible to estimate the error of the two detection units as in the above embodiment. For example, the first error estimating unit 207 obtains the true value of the shake based on the shake information and the motion vector detected by the first shake detecting unit 201, and calculates the true value of the shake and the first shake detecting unit 201. The correction error (detection error) of the first blur correction unit 107 is estimated based on the shake information (observed value) detected by the shake correction unit 107 . Further, the first error estimating unit 207 obtains the true value of the shake based on the shake information and the motion vector detected by the shake detecting unit 201 of the first shake detecting unit 201, and the true value of the shake and the second The correction error (detection error) of the second shake correction unit 108 is estimated based on the shake information (observed value) detected by the shake detection unit 204 .

(Other examples)
The present invention provides a system or device with a program that implements one or more of the functions of the above-described embodiments via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the invention.

107 First blur correction section 108 Second blur correction section 201 First blur detection section 204 Second blur detection section 207 First error estimation section 208 Second error estimation section

Claims (29)

An imaging device with a detachable lens device,
a vector detection unit that detects motion vectors from multiple images;
a first shake detection unit that detects shake information of the imaging device;
A first image stabilization unit that acquires a first shake correction amount based on first shake information detected by the first shake detection unit, and corrects image blur based on the acquired first shake correction amount. a blur correction section;
an error estimation unit that estimates an error included in the first shake information detected by the first shake detection unit;
a communication means for receiving a second blur correction amount, which is a blur correction amount of a second blur correction unit included in the lens device, from the lens device;
The error estimator is configured to calculate the first shake information detected by the first shake detector based on the motion vector detected by the vector detector and the second shake correction amount received by the communication means. Estimate the error included in
The image capturing apparatus is characterized in that the first blur correction section obtains the first blur correction amount based on the estimation result by the error estimation section. The error estimator includes:
Third shake information is acquired based on the motion vector and the second shake correction amount received by the communication means, and third shake information is obtained based on the third shake information and the shake correction amount detected by the first shake detection unit. The imaging apparatus according to claim 1, wherein an error included in the first shake information detected by the first shake detector is estimated based on the first shake information. The error estimation unit is characterized in that the third shake information is acquired based on the sum of the remaining shake correction amount based on the motion vector and the second shake correction amount received by the communication means. The imaging device according to claim 2. An imaging device with a detachable lens device,
a vector detection unit that detects motion vectors from multiple images;
a first shake detection unit that detects shake information of the imaging device;
A first image stabilization unit that acquires a first shake correction amount based on first shake information detected by the first shake detection unit, and corrects image blur based on the acquired first shake correction amount. a blur correction section;
an error estimation unit that estimates an error included in the first shake information;
The motion vector and the first blur correction amount are transmitted to the lens device, and third blur information acquired by the lens device based on the motion vector and the first blur correction amount is transmitted to the lens device. a communication means for receiving from the
The error estimation unit estimates an error included in the first shake information detected by the first shake detection unit based on the third shake information received from the lens device,
The image capturing apparatus is characterized in that the first blur correction section obtains the first blur correction amount based on the estimation result by the error estimation section. further comprising a phase compensation unit that performs phase compensation on the motion vector and the first blur correction amount,
The communication means transmits the motion vector on which the phase compensation has been performed and the first blur correction amount to the lens device, and transmits the motion vector on which the phase compensation has been performed and the first blur correction amount. The imaging device according to claim 4, wherein third shake information based on the following is received from the lens device. An imaging device with a detachable lens device,
a vector detection unit that detects motion vectors from multiple images;
a first shake detection unit that detects shake information of the imaging device;
A first image stabilization unit that acquires a first shake correction amount based on first shake information detected by the first shake detection unit, and corrects image blur based on the acquired first shake correction amount. a blur correction section;
an error estimation unit that acquires third shake information based on the motion vector and the first shake correction amount, and estimates an error included in the first shake information based on the third shake information; ,
An imaging device comprising: communication means for transmitting the third shake information to the lens device. comprising a phase compensation unit that performs phase compensation on the third shake information,
The imaging device according to claim 6, wherein the communication means transmits the third shake information on which the phase compensation has been performed to the lens device. 8. The error estimation unit acquires the third shake information based on the sum of the remaining shake correction amount based on the motion vector and the first shake correction amount. The imaging device described. An imaging system comprising an imaging device and a lens device detachable from the imaging device,
The imaging device includes:
a vector detection unit that detects motion vectors from multiple images;
a first shake detection unit that detects shake information of the imaging device;
A first image stabilization unit that acquires a first shake correction amount based on first shake information detected by the first shake detection unit, and corrects image blur based on the acquired first shake correction amount. a blur correction section;
a first communication means for communicating with the lens device,
The lens device includes:
a second shake detection unit that detects shake information of the lens device;
A second shake correction amount is acquired based on second shake information detected by the second shake detection unit, and image blur is corrected based on the acquired second shake correction amount. a blur correction section;
a second communication means for communicating with the imaging device,
The imaging system is
Shake information of the imaging system is acquired based on either the first blur correction amount or the second blur correction amount and the motion vector, and the shake information of the imaging system is acquired based on the shake information of the imaging system. An imaging system comprising: an error estimator that estimates an error included in the shake information and an error included in the second shake information. The error estimating unit sets a priority to determine which error included in the second shake detecting unit or the first shake detecting unit should be estimated preferentially. The imaging system according to item 9. 11. The imaging system according to claim 10, wherein the error estimation section sets the priority according to detection accuracy of the first shake detection section and the second shake detection section. 12. The imaging system according to claim 11, wherein the error estimating unit sets a high priority to a shake detecting unit having a high detection accuracy. 11. The imaging system according to claim 10, wherein the error estimating unit sets the priority of the shake detecting unit having a large error in shake information to be high. 11. The imaging system according to claim 10, wherein the error estimating unit sets a high priority to a shake detecting unit having a large error deviation in shake information. 11. The error estimation section sets the priority of the shake detection section according to the correction performance of the first shake correction section and the second shake correction section. Imaging system. 16. The imaging system according to claim 15, wherein the error estimating unit sets a high priority to a shake detecting unit paired with the shake correcting unit whose frequency of shake to be corrected is low. 17. The imaging system according to claim 10, wherein the error estimation section alternately estimates the error of the first shake detection section and the second shake detection section. A lens device that can be attached to and detached from an imaging device,
a second shake detection unit that detects shake information of the lens device;
A second shake correction amount is acquired based on second shake information detected by the second shake detection unit, and image blur is corrected based on the acquired second shake correction amount. a blur correction section;
an error estimation unit that estimates an error included in the second shake information detected by the second shake detection unit;
a communication means for receiving motion vector information acquired by the imaging device from the imaging device,
The error estimation unit estimates an error included in the second shake information based on the motion vector information received by the communication means and the second shake correction amount,
The lens device, wherein the second blur correction section obtains the second blur correction amount based on the estimation result by the error estimation section. The error estimation unit acquires third shake information based on the motion vector information and the second shake correction amount received by the communication means, and calculates the third shake information and the second shake information. 19. The lens device according to claim 18, wherein an error included in the second shake information is estimated based on the following. 20. The error estimation unit obtains the third shake information based on a sum of a remaining shake correction amount based on the motion vector information and the second shake correction amount. lens device. 21. The lens device according to claim 19, wherein the communication means transmits the third shake information to the imaging device. The communication means receives, from the imaging device, first shake information detected by a first shake detection unit included in the imaging device,
The error estimation unit estimates an error included in the first shake information based on the third shake information and the first shake information, and transmits the estimation result to the imaging device. The lens device according to claim 19 or 20. A lens device that can be attached to and detached from an imaging device,
a second shake detection unit that detects shake information of the lens device;
A second shake correction amount is acquired based on second shake information detected by the second shake detection unit, and image blur is corrected based on the acquired second shake correction amount. a blur correction section;
an error estimation unit that estimates an error included in the second shake information;
Communication for transmitting the second blur correction amount to the imaging device and receiving third blur information acquired by the imaging device based on the second blur correction amount and motion vector information from the imaging device. equipped with the means and
The error estimation unit estimates an error included in the second shake information based on the third shake information received by the communication means,
The lens device is characterized in that the second blur correction section obtains the blur correction amount based on the estimation result by the error estimation section. A lens device that can be attached to and detached from an imaging device,
a second shake detection unit that detects shake information of the lens device;
A second shake correction amount is acquired based on second shake information detected by the second shake detection unit, and image blur is corrected based on the acquired second shake correction amount. a blur correction section;
an error estimation unit that estimates an error included in the second shake information;
a communication means for receiving a first shake correction amount that is a correction amount of a first shake correction unit included in the image pickup device and motion vector information from the image pickup device;
The error estimation unit acquires third shake information based on the first shake correction amount and the motion vector information, and calculates an error included in the second shake information based on the third shake information. Estimate
The lens device, wherein the second blur correction section obtains the second blur correction amount based on the estimation result by the error estimation section. An imaging device with a detachable lens device,
a vector detection unit that detects motion vectors from multiple images;
a first shake detection unit that detects shake information of the imaging device;
A first image stabilization unit that acquires a first shake correction amount based on first shake information detected by the first shake detection unit, and corrects image blur based on the acquired first shake correction amount. a blur correction section;
communication means for receiving second shake information that is shake information of the lens device from the lens device;
an error estimation unit that estimates an error included in the second shake information based on the second shake information received by the communication means,
The error estimation unit acquires third shake information based on the motion vector detected by the vector detection unit and the first shake information, and the third shake information and the second shake information received by the communication means. Estimating the error included in the second shake information based on the shake information,
The imaging device is characterized in that the communication means transmits an estimation result of the error included in the second shake information by the error estimator to the lens device. A method for controlling an imaging device having a detachable lens device, the method comprising:
a vector detection step of detecting motion vectors from multiple images;
a shake detection step of detecting first shake information by a first shake detection section that detects shake information of the imaging device;
A first shake correction amount is acquired based on the first shake information detected by the first shake detection unit, and image blur is corrected based on the acquired first shake correction amount. a correction process;
an error estimation step of estimating an error included in the first shake information detected by the first shake detector;
a communication step of receiving a second blur correction amount, which is a blur correction amount of a second blur correction unit included in the lens device, from the lens device;
The error estimation step includes determining the first shake information detected by the first shake detection unit based on the motion vector detected in the vector detection step and the second shake correction amount received in the communication step. Estimate the error included in
A control method characterized in that, in the blur correction step, the first blur correction amount is obtained based on the estimation result in the error estimation step. A method for controlling a lens device detachable from an imaging device, the method comprising:
a shake detection step of detecting second shake information by a second shake detection section that detects shake information of the lens device;
A second shake correction amount is acquired based on the second shake information detected by the second shake detection unit, and image blur is corrected based on the second shake correction amount obtained. a correction process;
an error estimation step of estimating an error included in the second shake information detected by the second shake detector;
a communication step of receiving motion vector information acquired by the imaging device from the imaging device,
The error estimation step estimates an error included in the second shake information based on the motion vector information and the second shake correction amount received in the communication step,
A control method characterized in that, in the blur correction step, the second blur correction amount is obtained based on the estimation result in the error estimation step. A method for controlling an imaging device having a detachable lens device, the method comprising:
a vector detection step of detecting motion vectors from multiple images;
a shake detection step of detecting first shake information by a first shake detection section that detects shake information of the imaging device;
A first shake correction amount is acquired based on the first shake information detected by the first shake detection unit, and image blur is corrected based on the acquired first shake correction amount. a correction process;
a first communication step of receiving second shake information that is shake information of the lens device from the lens device;
an error estimation step of estimating an error included in the second shake information based on the second shake information received in the first communication step;
a second communication step of transmitting an error estimation result included in the second shake information obtained by the error estimation step to the lens device;
The error estimation step acquires third shake information based on the motion vector detected in the vector detection step and the first shake information, and acquires third shake information based on the third shake information and the first shake information received in the first communication step. A control method characterized in that an error included in the second shake information is estimated based on the second shake information. A method for controlling an imaging system including an imaging device and a lens device detachable from the imaging device, the method comprising:
a vector detection step of detecting motion vectors from multiple images;
In the imaging device, a first shake detection step of detecting shake information of the imaging device;
In the lens device, a second shake detection step of detecting shake information of the lens device;
A first blur correction amount is acquired based on the first shake information detected in the first shake detection step, and image blur is corrected based on the acquired first shake correction amount. blur correction process,
A second blur correction amount is acquired based on the second shake information detected in the second shake detection step, and a second blur correction amount is corrected based on the obtained second shake correction amount. blur correction process,
Shake information of the imaging system is acquired based on either the first blur correction amount or the second blur correction amount and the motion vector, and the shake information of the imaging system is acquired based on the shake information of the imaging system. A control method comprising: an error estimation step of estimating an error included in the shake information and an error included in the second shake information.

Images