WO2019104622A1 - 差异校准方法、双目视觉系统和计算机可读存储介质 differential calibration method, binocular vision system, and computer-readable storage medium - Google Patents

Abstract

Disclosed is a differential calibration method. The differential calibration method can be implemented by a binocular vision system (10). The binocular vision system (10) comprises a first camera (12) and a second camera (14). The differential calibration method comprises: (S1) acquiring in real-time an image brightness value of a first image and that of a second image of a same scene captured at the current moment correspondingly by the first camera (12) and by the second camera (14); (S2) calculating the image brightness difference between the first image and the second image; (S3) determining whether the image brightness difference is greater than a predetermined threshold; and, (S4) when the image brightness difference is greater than the predetermined threshold, updating an exposure parameter for the next moment of the first camera (12) and/or that of the second camera (14). Also disclosed are the binocular vision system (10) and the computer-readable storage medium.

Description

Differential calibration method, binocular vision system, and computer readable storage medium Technical field

The present invention relates to the field of electronic technologies, and in particular, to a differential calibration method, a binocular vision system, and a computer readable storage medium.

Background technique

In the related art binocular vision system, two cameras are used to acquire the left eye image and the right eye image of the same scene, and if there is a difference in brightness between the left eye image and the right eye image collected by the two cameras, the binocular vision system measurement may be caused ( Distance) failure or erroneous measurement.

Summary of the invention

Embodiments of the present invention provide a differential calibration method, a binocular vision system, and a computer readable storage medium.

A difference calibration method according to an embodiment of the present invention is used in a binocular vision system, the binocular vision system includes a first camera and a second camera, and the difference calibration method includes:

Acquiring image brightness values of the first image and the second image of the same scene currently acquired by the first camera and the second camera in real time;

Calculating an image brightness difference value between the first image and the second image;

Determining whether the image brightness difference value is greater than a predetermined threshold; and

And updating an exposure parameter of the first camera and/or the second camera at a next moment when the image brightness difference value is greater than the predetermined threshold.

A binocular vision system according to an embodiment of the present invention includes a first camera, a second camera, and a processor, wherein the processor is configured to:

Acquiring image brightness values of the first image and the second image of the same scene currently acquired by the first camera and the second camera in real time;

Calculating an image brightness difference value between the first image and the second image;

Determining whether the image brightness difference value is greater than a predetermined threshold; and

And updating an exposure parameter of the first camera and/or the second camera at a next moment when the image brightness difference value is greater than the predetermined threshold.

A computer readable storage medium in accordance with an embodiment of the present invention includes a computer program for use in conjunction with a binocular vision system, the computer program being executable by a processor to perform the difference calibration method of the above-described embodiments.

The difference calibration method, the binocular vision system, and the computer readable storage medium of the embodiments of the present invention are detected by real time The image brightness difference value of the first image and the second image is used to update the exposure parameters of the first camera and the second camera when the image brightness difference value is large, so that the brightness difference values of the collected first image and the second image are controlled at a certain value Within the range, thereby reducing the measurement failure or erroneous measurement caused by the difference in image brightness of the two images acquired by the first camera and the second camera at the same time in the image processing. The additional aspects and advantages of the embodiments of the present invention will be set forth in part in the description which follows.

DRAWINGS

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from

1 is a functional block diagram of a binocular vision system in accordance with some embodiments of the present invention.

2 is a flow diagram of a differential calibration method in accordance with some embodiments of the present invention.

3 is a flow diagram of a differential calibration method in accordance with some embodiments of the present invention.

4 is a flow diagram of a differential calibration method in accordance with some embodiments of the present invention.

5 is a flow diagram of a differential calibration method in accordance with some embodiments of the present invention.

6 is a schematic diagram of image region division of a difference calibration method according to some embodiments of the present invention.

7 is a schematic diagram of image region division of a difference calibration method according to some embodiments of the present invention.

8 is a flow diagram of a differential calibration method in accordance with some embodiments of the present invention.

9 is a flow diagram of a differential calibration method in accordance with some embodiments of the present invention.

10 is a flow diagram of a differential calibration method in accordance with some embodiments of the present invention.

11 is a flow diagram of a differential calibration method in accordance with some embodiments of the present invention.

Detailed ways

The embodiments of the present invention are described in detail below, and the examples of the embodiments are illustrated in the drawings, wherein the same or similar reference numerals indicate the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the drawings are intended to be illustrative of the invention and are not to be construed as limiting.

In the description of the present invention, it is to be understood that the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" or "second" may include one or more of the described features either explicitly or implicitly. In the description of the present invention, the meaning of "a plurality" is two or more unless specifically and specifically defined otherwise.

In the description of the present invention, it should be noted that the terms "installation", "connected", and "connected" are to be understood broadly, and may be fixed or detachable, for example, unless otherwise explicitly defined and defined. Connected, or connected in one piece; can be mechanical, electrical, or can communicate with each other; either directly or through an intermediary Indirectly connected, it can be the internal communication of two components or the interaction of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood on a case-by-case basis.

The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and arrangements of the specific examples are described below. Of course, they are merely examples and are not intended to limit the invention. In addition, the present invention may be repeated with reference to the numerals and/or reference numerals in the various examples, which are for the purpose of simplicity and clarity, and do not indicate the relationship between the various embodiments and/or arrangements discussed. Moreover, the present invention provides examples of various specific processes and materials, but one of ordinary skill in the art will recognize the use of other processes and/or the use of other materials.

Referring to FIG. 1, a binocular vision system 10 of an embodiment of the present invention includes a first camera 12, a second camera 14, and a processor 16. The processor 16 is configured to acquire the image brightness values of the first image and the second image of the same scene at the current time by the first camera 12 and the second camera 14 in real time. The processor 16 is configured to calculate an image brightness difference value between the first image and the second image. The processor 16 is configured to determine whether the image brightness difference value is greater than a predetermined threshold. The processor 16 is configured to update the exposure parameters of the first camera 12 and/or the second camera 14 at the next moment when the image brightness difference value is greater than a predetermined threshold.

Referring to FIG. 2, the difference calibration method of the embodiment of the present invention can be applied to the binocular vision system 10 of the embodiment of the present invention, that is, the binocular vision system 10 of the embodiment of the present invention can apply the difference of the embodiment of the present invention. The calibration method updates the exposure parameters of the first camera 12 and/or the second camera 14. The differential calibration method includes the following steps:

S1, real-time acquiring image brightness values of the first image and the second image of the same scene acquired by the first camera 12 and the second camera 14 at a time;

S2, calculating an image brightness difference value between the first image and the second image;

S3, determining whether the image brightness difference value is greater than a predetermined threshold; and

S4. Update the exposure parameters of the first camera 12 and/or the second camera 14 at the next moment when the image brightness difference value is greater than a predetermined threshold.

The binocular vision system 10 of the embodiment of the present invention simultaneously acquires digital images of the same scene from different angles by two camera modules mounted at fixed positions based on the principle of parallax to obtain three-dimensional shape and position information of the scene. In some embodiments, the binocular vision system 10 can be applied to devices such as drones, smart robots, driverless cars, and panoramic depth cameras to achieve perception of the three-dimensional shape of the scene around the device and measurement of the positional distance.

It can be understood that there are often system differences between the two camera modules of the binocular vision system 10. Therefore, the binocular vision system 10 requires that system differences between the two camera modules be minimized, wherein two camera modules are collected. Minimizing the difference in image brightness between the images is advantageous for improving the accuracy of the measurement results of the binocular vision system 10. Therefore, it is necessary to calibrate the brightness of the image acquired by the two cameras, and reduce or eliminate system differences by compensating for the difference value of the calibration. When the binocular vision system 10 is in a scene environment with varying brightness, the difference values that need to be compensated under different brightness conditions may be different. As such, the binocular vision system 10 and the difference calibration method of the embodiment of the present invention detect the luminance difference value of the first image and the second image in real time, and update the first camera 12 and/or the second camera 14 when the luminance difference value is large. The exposure parameters for the next moment. The binocular vision system 10 can update the exposure parameters of the first camera 12 and/or the second camera 14 at the next moment when the brightness value of the scene in which the image is acquired changes, so that the first camera 12 and the second camera 14 are next. The image brightness difference between the first image and the second image of the same scene is minimized at all times. Preferably, the first camera 12 and the second camera 14 are used to acquire the image brightness of the first image and the second image of the same scene at the next moment. Consistently, it is advantageous to reduce measurement failure or erroneous measurement due to image brightness difference in image processing.

The binocular vision system 10 captures a digital image of the current scene through the first camera 12 and the second camera 14, wherein the first camera 12 captures the first image of the current scene from one angle while the second camera 14 captures the same image from another angle The second image of the scene. The processor 16 may implement step S1 to acquire image brightness values of the first image and the second image in real time. The processor 16 may implement step S2 to calculate an image brightness difference value by a preset algorithm according to image brightness values of the first image and the second image. The processor 16 may implement step S3 to compare the image brightness difference value with a predetermined threshold value to determine whether the image brightness difference value exceeds a predetermined threshold. It can be understood that when the image brightness difference value is less than or equal to a predetermined threshold, the image brightness difference between the first image and the second image has little influence on the accuracy of the measurement result in the subsequent image processing. At this time, the first camera does not need to be updated. 12 and exposure parameters of the second camera 14. The processor 16 may implement step S4. When the image brightness difference value is greater than a predetermined threshold, the image brightness difference between the first image and the second image is more likely to cause measurement failure or erroneous measurement in subsequent image processing. The processor 16 updates the exposure parameters of the first camera 12 and/or the second camera 14 at the next moment, so that the image brightness difference between the first image and the second image acquired after the update is reduced, which is beneficial to improve binocular vision. System 10 measures the accuracy of the results.

Referring to FIG. 3, in some embodiments, the difference calibration method includes: S5, determining whether the binocular vision system 10 continues to work when the image brightness difference value is less than or equal to a predetermined threshold, and continues to work in the binocular vision system 10. At the same time, the image brightness values of the first image and the second image of the current scene acquired by the first camera 12 and the second camera 14 at the next moment are acquired.

It can be understood that the processor 16 can implement step S5. During the operation of the binocular vision system 10, the first camera 12 and the second camera 14 continuously collect images of the current scene. When the image brightness difference value is less than or equal to a predetermined threshold, it is not necessary to update the exposure parameters of the binocular vision system 10. At this time, the processor 16 acquires the first image and the second image at the next moment to detect the brightness difference of the system in real time.

In some embodiments, the difference calibration method includes: S5, after updating the exposure parameters of the first camera 12 and/or the second camera 14, determining whether the binocular vision system 10 continues to operate and continuing in the binocular vision system 10 In operation, the image brightness values of the first image and the second image of the current scene acquired by the first camera 12 and the second camera 14 at the next moment are acquired.

In this way, the processor 16 can implement step S5. After the update is completed, the exposure parameters of the first camera 12 and the second camera 14 at the next moment are updated exposure parameters, and the images of the first image and the second image are acquired thereby. The decrease in the brightness difference value is advantageous for improving the accuracy of the measurement result of the binocular vision system 10. At this time, the processor 16 acquires the first image and the second image at the next moment to detect the image brightness difference of the system in real time.

Referring to FIG. 4, in some embodiments, step S1 includes:

S12. Acquire a region division of the first image and the second image and a weight of the corresponding region; and

S14. Calculate the brightness values of the first image and the second image according to the area brightness value of the area division and the weight of the corresponding area, respectively.

As such, the processor 16 may implement steps S12 and S14, and the image brightness values of the first image and the second image may be weighted brightness values, and the processor 16 may divide the image into a plurality of regions and divide according to the location of the region of interest. The corresponding weights are set for each area. Specifically, in some embodiments, the weight of the region of interest is the largest, and the weights of other regions in the image that are far from the region of interest are gradually reduced. As such, the image brightness values of the first image and the second image are greatly affected by the brightness values of the region of interest, which is advantageous for improving the accuracy of the measurement results of the binocular vision system 10 in the image region of interest. It can be understood that the region of interest can be determined according to the focus region. Of course, the region of interest can also be obtained by other implementations.

Referring to FIG. 5, in some embodiments, step S12 includes:

S122. Acquire an operating state of the binocular vision system 10;

S124. Determine an area of interest of the first image and the second image according to the working state; and

S126. Acquire a region division of the first image and the second image and a weight of the corresponding region according to the region of interest.

It can be understood that the binocular vision system 10 can be applied in different devices, and correspondingly, the regions of interest of the images can be different. For example, in a drone, the binocular vision system 10 is often in high altitude when it is running, and it is necessary to detect and notice whether there is an obstacle above the scene, so that the region of interest can be the upper half of the image; and in the driverless car. The binocular vision system 10 is often located on the ground during operation, and needs to detect three-dimensional information of the road surface and distance information. Thus, the region of interest may be the lower half of the image. The binocular vision system 10 can select different region divisions and weights of corresponding regions according to different regions of interest.

Similarly, when the binocular vision system 10 is in different operating states, the corresponding regions of interest of the images may be different. For example, in an unmanned vehicle, when the driverless car turns left, in addition to detecting the three-dimensional information of the road surface and the distance information, it is also necessary to detect the left road condition information, and the corresponding interest area may be the lower left part of the image, and the corresponding area The weight size distribution is lower left > lower right > upper left and upper right. When the driverless car turns right, in addition to detecting the three-dimensional information and distance information of the road surface, it is also necessary to detect the road condition information on the right side. The corresponding interest area may be the lower left part of the image, and the weight distribution of the corresponding area is lower right> lower left. > Top left, top right.

As such, the processor 16 can implement step S122, step S124, and step S126, according to the binocular vision system 10 The working state determines an area of interest of the image, and obtains the area division of the image and the weight of the corresponding area according to the area of interest to calculate the image brightness value.

In some embodiments, processor 16 pre-stores the correspondence of regions of interest to region partitions and weight distributions. In this way, after determining the region of interest, the region division can be acquired according to the correspondence relationship and the weighted luminance value of the image can be calculated.

In some embodiments, the region partitioning of the binocular vision system 10 can divide the image equally into a plurality of regions of the same size. The region of interest shown in FIG. 6 is the central region of the image, and the image is equally divided into 4*4 regions of the same size, and the number in the region is the weight of the corresponding region.

In some embodiments, the area division of the binocular vision system 10 may be smaller in the area of interest region division, and the area divided away from the interest area may be larger. The interest area shown in FIG. 7 is the lower left area of the image, and the lower left area of the image is divided into 4*4 small areas of the same size, and the upper left area and the lower right area of the image are divided into 3*3 small areas of the same size, images. The upper right area is divided into 2*2 small areas of the same size, and the values in each small area are the weights of the corresponding areas. As such, the accuracy of the binocular vision system 10 in the region of interest can be improved.

Specifically, the weight of the corresponding area when the image area is divided can be flexibly configured according to requirements.

Referring to FIG. 8, in some embodiments, step S14 includes:

S142. Calculate, respectively, a sum of products of luminance values of respective regions and weights of corresponding regions to obtain total weight luminance values of the first image and the second image; and

S144. Calculate a ratio of the total weight luminance value to the total weight to obtain image luminance values of the first image and the second image, respectively.

As such, the processor 16 may implement steps S142 and S144 to calculate image brightness values of the first image and the second image based on the respective region luminance values and corresponding weights. For example, the luminance values of the respective regions of the image are a1, a2, a3, ..., an, and the weights of the corresponding regions are k1, k2, k3, ..., kn, respectively, and the total weight value calculated in step S142 is A = k1 * a1. +k2*a2+k3*a3+...+kn*an. The image luminance value a = (k1 * a1 + k2 * a2 + k3 * a3 + ... + kn * an) / (k1 + k2 + k3 + ... + kn) calculated in step S144.

Referring to FIG. 9, in some embodiments, step S4 includes:

S42. Calculate a difference calibration parameter of the binocular vision system 10 according to the image brightness value of the first image and the second image and the reference brightness value when the image brightness difference value is greater than a predetermined threshold; and

S44. Update the exposure parameters of the first camera 12 and/or the second camera 14 at the next moment according to the difference calibration parameter and the current exposure parameter of the first camera 12 or the second camera 14.

As such, when the processor 16 determines that the exposure parameters of the first camera 12 and the second camera 14 need to be updated, the exposure parameters can be calibrated for updating by the difference calibration parameters. The difference calibration parameter is a difference coefficient between the exposure parameter of the first camera 12 and/or the second camera 14 that is not calibrated and the updated exposure parameter. The processor 16 may implement step S42 to calculate a difference calibration parameter according to a preset algorithm when the image brightness difference value is greater than a predetermined threshold. The processor 16 may implement step S44 to update the exposure parameters of the first camera 12 and/or the second camera 14 at the next moment based on the calculated difference calibration parameters.

In some embodiments, in some embodiments, the reference brightness value is any of an image brightness value of the first image, an image brightness value of the second image, or an average image brightness value of the first image and the second image. One.

It can be understood that in order to minimize the difference in brightness between the first image and the second image, the binocular vision system 10 needs to select a reference brightness value and update the first camera 12 and/or the second camera 14 with reference to the reference brightness value. The exposure parameter at a moment makes the luminance value of the acquired first image coincide with the image luminance value of the second image. When the image brightness difference value is greater than a predetermined threshold, the processor 16 generates a reference brightness value according to a preset algorithm according to the image brightness values of the first image and the second image, such that the reference brightness and the currently acquired brightness of the first image and the second image Correspondingly, it is advantageous to update the exposure parameters of the first camera 12 and the second camera 14 in real time and accurately.

Specifically, the reference brightness value may be an image brightness value of the first image, and when the image brightness difference value is greater than a predetermined threshold, the exposure parameter of the next time of the second camera 14 is updated to make the image brightness value of the second image acquired at the next time. Consistent with the image brightness value of the first image. Similarly, the reference brightness value may be an image brightness value of the second image, and when the image brightness difference value is greater than a predetermined threshold, the exposure parameter of the next time of the first camera 12 is updated to make the image brightness value of the first image acquired at the next time. Consistent with the image brightness value of the second image. Similarly, the reference brightness value may be an average image brightness value of the first image and the second image, and when the image brightness difference value is greater than a predetermined threshold, the exposure parameters of the next time of the first camera 12 and the second camera 14 are respectively updated to enable The image brightness value of the first image acquired at the next time is equal to the image brightness value of the second image and coincides with the average image brightness value of the first image and the second image.

In some embodiments, the average image brightness value of the first image and the second image may be any one of a weighted average brightness value and an arithmetic mean brightness value.

Thus, when the reference brightness generated by the processor 16 is the average image brightness value of the first image and the second image, it is necessary to simultaneously update the exposure parameters of the first camera 12 and the second camera 14 at the next time, and the parameter variation range is small.

Specifically, the weighted average luminance values of the first image and the second image may be (m*a+n*b)/(m+n), where a and b are image luminances of the first image and the second image, respectively. The values, m, n are weights corresponding to the image brightness values of the first image and the second image, respectively.

Specifically, the arithmetic average luminance value of the first image and the second image may be (a+b)/2, where a and b are image luminance values of the first image and the second image, respectively.

In some embodiments, the image brightness difference value includes an absolute difference value, and step S2 includes: calculating an absolute difference according to the image brightness value of the first image and the second image and the current difference calibration parameter corresponding to the acquired first image and the second image. The value of the absolute difference is calculated using the following conditional formula:

D=|a-(b/xb)*xa| or D=|(a/xa)*xb-b|;

Where D is the absolute difference value, a and b are the image brightness values of the first image and the second image, respectively, xa and xb respectively To collect current difference calibration parameters corresponding to the first image and the second image.

It can be understood that the processor 16 can detect the absolute difference values of the first image and the second image. In the case where the brightness is large, the absolute difference values are significantly different, which is advantageous for comparison with a predetermined threshold to determine whether the binocular vision system 10 needs to be updated. Exposure parameters.

In some embodiments, the image brightness difference value includes a relative difference value, and step S2 includes: calibrating according to the image brightness value of the first image and the second image, the reference brightness value, and the current difference corresponding to the acquired first image and the second image. The parameter calculates the relative difference value, wherein the relative difference value is the ratio of the absolute difference value to the reference brightness value, and the absolute difference value is calculated by the following conditional expression:

D=|a-(b/xb)*xa| or D=|(a/xa)*xb-b|;

Where D is an absolute difference value, a and b are image brightness values of the first image and the second image, respectively, and xa and xb are current difference calibration parameters corresponding to the first image and the second image, respectively.

It can be understood that the processor 16 can detect the relative difference values of the first image and the second image. If the brightness is small, the absolute difference values are not significantly different. At this time, detecting the relative difference value is beneficial to compare with a predetermined threshold to determine Whether the exposure parameters of the binocular vision system 10 need to be updated.

Referring to FIG. 10, in some embodiments, the predetermined threshold includes an absolute threshold and a relative threshold, and step S3 includes:

S32, determining whether the absolute difference value is greater than an absolute threshold; and

S34. Determine whether the relative difference value is greater than a relative threshold.

As such, the processor 16 may implement step S32 and step S34. When the absolute difference value is greater than the absolute threshold and/or the relative difference value is greater than the relative threshold, the image brightness difference value of the first image and the second image may be considered to be greater than a predetermined threshold. The exposure parameters of the first camera 12 and the second camera 14 should be updated to reduce measurement failure or erroneous measurements that occur in subsequent image processing.

Referring to FIG. 11 , in some embodiments, step S42 includes: S422, according to image brightness values of the first image and the second image, and current difference calibration parameters and reference brightness values corresponding to the first image and the second image. The difference calibration parameters of the first image and the second image are calculated, and the difference calibration parameters are calculated by the following conditional formula:

a/xa*aedc=ref/xref*aedc_ref(1) and/or b/xb*aedc=ref/xref*aedc_ref(1);

Where a and b are image brightness values of the first image and the second image, respectively, xa and xb are current difference calibration parameters corresponding to the first image and the second image, aedc is the difference calibration parameter, and ref is the reference. The brightness value, xref is the difference calibration parameter corresponding to the reference brightness value, and aedc_ref(1) is the reference difference calibration parameter. In this embodiment, both xref and aedc_ref(1) are 1.

It can be understood that the processor 16 can implement the step S422. During the continuous operation of the binocular vision system 10, the current difference calibration parameter may be a difference calibration parameter corresponding to the previous time when the exposure parameters of the first camera 12 and the second camera 14 are updated. The image brightness values of the first image and the second image may be brightness after calibration by the current difference calibration parameter value. The a/xa, b/xb may be considered as the image brightness values acquired by the first camera 12 and the second camera 14 that have not been calibrated by the current difference calibration parameter when the first image and the second image are currently acquired. In the above conditional expression, the difference calibration parameter xref and the reference difference calibration parameter aedc_ref(1) corresponding to the reference luminance value are both 1, that is, the right side of the conditional expression is the reference luminance ref, and thus, the processor 16 can calculate according to the above conditional expression. The difference calibration parameter aedc makes the image brightness value of the first image and/or the second image after calibration coincide with the reference brightness value.

In some embodiments, the exposure parameters include an automatic exposure parameter and a calibration exposure parameter, and step S44 includes: S442, calculating the first camera 12 and/or the first based on the difference calibration parameter and the exposure parameters of the first camera 12 and the second camera 14. The calibration exposure parameters of the two cameras 14.

It can be understood that the camera module can usually perform automatic exposure, and automatically adjust the exposure parameters according to the intensity of the light to prevent overexposure or deficiency. In this way, the binocular vision system 10 can determine whether it is necessary to update the exposure parameters of the first camera 12 and/or the second camera 14 based on the brightness value of the automatically exposed image, and the automatic exposure and the difference calibration parameters simultaneously act to improve the double The accuracy of the visual system 10 measurement results ensures that the binocular vision system 10 is operating normally.

Processor 16 may implement step S442 to calculate a calibration exposure parameter when it is desired to update the exposure parameters of first camera 12 and second camera 14. The processor 16 may update the exposure parameters of the first camera 12 and/or the second camera 14 to the calculated calibration exposure parameters.

In some embodiments, the auto exposure parameter may be an exposure parameter obtained by the first camera 12 and/or the second camera 14 automatically exposing according to the image brightness values of the respective acquired images.

In some embodiments, the auto exposure parameter may be an exposure parameter obtained by the first camera 12 and/or the second camera 14 automatically exposing according to a reference brightness value.

In some embodiments, the auto exposure parameter includes at least one of a coarse adjustment exposure time, an analog gain, and a digital gain. The calibration exposure parameters include at least one of a coarse adjustment exposure time, a fine adjustment exposure time, a line cycle time, an analog gain, and a digital gain.

It can be understood that the automatic exposure generally adjusts the coarse adjustment exposure time, the analog gain and the digital gain of the camera module, and the automatic exposure parameter of the binocular vision system 10 can be at least one of a coarse adjustment exposure time, an analog gain, and a digital gain. The calibration exposure parameter is adjusted based on the automatic exposure, and the adjusted parameters may be at least one of a coarse adjustment exposure time, a fine adjustment exposure time, a line cycle time, an analog gain, and a digital gain.

In some embodiments, when the calculated calibration exposure parameter is completely different from the type of the automatic exposure parameter, in step S442, the calibration exposure parameter is calculated by the following conditional expression:

Y=r*c;

Where Y is the calibration exposure parameter, r is the default exposure parameter of the first camera 12 or the second camera 14, and c is the difference calibration parameter.

It can be understood that when the binocular vision system 10 needs to update the exposure parameters of the first camera 12 and/or the second camera 14, the calibration exposure parameters calculated by the processor 16 are all different from the types of the automatic exposure parameters, that is, the parameters adjusted during the automatic exposure. There is no effect on the exposure parameters that need to be updated. As such, the calibration exposure parameters are calculated and updated directly from the difference calibration parameters and the default exposure parameters. The default exposure parameter is a parameter value of each exposure parameter of the image acquired when the camera does not adopt the difference calibration parameter.

In some embodiments, when the calculated calibration exposure parameters are all the same as the types of the automatic exposure parameters, in step S442, the calibration exposure parameters are calculated using the following conditional formula:

Y=g*c;

Wherein Y is a calibration exposure parameter, g is an automatic exposure parameter corresponding to the acquisition of the first image and/or the second image, and c is a difference calibration parameter.

It can be understood that when the binocular vision system 10 needs to update the exposure parameters of the first camera 12 and/or the second camera 14, the calibration exposure parameters calculated by the processor 16 are all the same as the types of the automatic exposure parameters, that is, the updated exposure parameters are automatically The exposure adjustment and the adjustment of the difference calibration parameters enable the luminance values of the first image and/or the second image to coincide with the reference luminance values. Thus, the calibration exposure parameters are calculated and updated based on the automatic exposure parameters based on the automatic exposure parameters and the difference calibration parameters.

In some embodiments, when the calculated calibration exposure parameter is partially the same as the type of the automatic exposure parameter and the other portion is different, in step S442, the calibration exposure parameter is calculated using the following conditional formula:

Y1=(g*c1) and Y2=(r*c2);

Wherein, Y1 and Y2 are calibration exposure parameters, g is an automatic exposure parameter corresponding to the first image and/or the second image, and r is a default exposure parameter of the first camera 112 or the second camera 114, c1*c2=c And c is the difference calibration parameter.

It can be understood that when the binocular vision system 10 needs to update the exposure parameters of the first camera 12 and/or the second camera 14, the calibration exposure parameters calculated by the processor 16 are partially the same as the types of the automatic exposure parameters and the other portions are different, ie, updated. The part of the exposure parameter whose calibration exposure parameter is the same as the type of the automatic exposure parameter is adjusted by the automatic exposure adjustment and the difference calibration parameter. The different parts of the calibration exposure parameter and the type of the automatic exposure parameter are adjusted by the difference calibration parameter, and the two parts work together to make adjustment. The image brightness value of the first image and/or the second image coincides with the reference brightness value. Thus, the portion of the calibration exposure parameter and the type of the automatic exposure parameter, that is, the Y1 portion, is calculated and updated based on the automatic exposure based on the automatic exposure parameter and the difference calibration parameter. The part where the calibration exposure parameter is different from the type of the automatic exposure parameter, that is, the Y2 part, is calculated and updated according to the difference calibration parameter and the default exposure parameter. Specifically, c1 and c2 in the above conditional formula can be flexibly configured according to requirements.

A computer readable storage medium in accordance with an embodiment of the present invention includes a computer program for use with a binocular vision system that can be executed by processor 16 to perform the differential calibration method of the above-described embodiments.

For example, a computer program can be executed by processor 16 to perform the difference calibration method described in the following steps:

Acquiring image brightness values of the first image and the second image of the same scene currently acquired by the first camera 12 and the second camera 14 in real time;

Calculating an image brightness difference value between the first image and the second image;

Determining whether the image brightness difference value is greater than a predetermined threshold; and

The exposure parameters of the first camera 12 and/or the second camera 14 at the next moment are updated when the image brightness difference value is greater than a predetermined threshold.

In the description of the present specification, reference is made to the terms "some embodiments", "one embodiment", "some embodiments", "illustrative embodiments", "example", "specific examples", or "some examples", etc. The descriptions of the specific features, structures, materials or features described in connection with the embodiments or examples are included in at least one embodiment or example of the invention. In the present specification, the schematic representation of the above terms does not necessarily mean the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a particular logical function or process. And the scope of the preferred embodiments of the invention includes additional implementations, in which the functions may be performed in a substantially simultaneous manner or in an opposite order depending on the functions involved, in the order shown or discussed. It will be understood by those skilled in the art to which the embodiments of the present invention pertain.

The logic and/or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer readable medium, Used in conjunction with, or in conjunction with, an instruction execution system, apparatus, or device (eg, a computer-based system, a system including a processor, or other system that can fetch instructions and execute instructions from an instruction execution system, apparatus, or device) Or use with equipment. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport a program for use in an instruction execution system, apparatus, or device, or in conjunction with the instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) having one or more wires, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM). In addition, the computer readable medium may even be a paper or other suitable medium on which the program can be printed, as it may be optically scanned, for example by paper or other medium, followed by editing, interpretation or, if appropriate, other suitable The method is processed to obtain the program electronically and then stored in computer memory.

It should be understood that portions of the invention may be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods may be implemented by software stored in a memory and executed by a suitable instruction execution system or Firmware to achieve. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques well known in the art: having logic gates for implementing logic functions on data signals. Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

A person skilled in the art can understand that all or part of the steps carried in implementing the above implementation method can be completed by a program to instruct related hardware, and the program can be stored in a computer readable storage medium, and the program is executed. Including one or a combination of the steps of the method embodiments.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. The integrated modules, if implemented in the form of software functional modules and sold or used as stand-alone products, may also be stored in a computer readable storage medium.

The above mentioned storage medium may be a read only memory, a magnetic disk or an optical disk or the like. Although the embodiments of the present invention have been shown and described, it is understood that the above-described embodiments are illustrative and are not to be construed as limiting the scope of the invention. The embodiments are subject to variations, modifications, substitutions and variations.

Claims (25)

1. A difference calibration method for a binocular vision system, wherein the binocular vision system comprises a first camera and a second camera, and the difference calibration method comprises:

   Acquiring image brightness values of the first image and the second image of the same scene currently acquired by the first camera and the second camera in real time;

   Calculating an image brightness difference value between the first image and the second image;

   Determining whether the image brightness difference value is greater than a predetermined threshold; and

   And updating an exposure parameter of the first camera and/or the second camera at a next moment when the image brightness difference value is greater than the predetermined threshold.
2. The difference calibration method according to claim 1, wherein the step of acquiring the image brightness values of the first image and the second image of the current scene acquired by the first camera and the second camera in real time comprises:

   Obtaining a region division of the first image and the second image and a weight of the corresponding region; and

   And calculating image brightness values of the first image and the second image according to the area brightness value of the area and the weight of the corresponding area.
3. The difference calibration method according to claim 2, wherein the step of acquiring the area division of the first image and the second image and the weight of the corresponding area comprises:

   Obtaining an operating state of the binocular vision system;

   Determining an area of interest of the first image and the second image according to the working state; and

   Acquiring the weight of the region division and the corresponding region of the first image and the second image according to the region of interest.
4. The difference calibration method according to claim 2, wherein the luminance values of the first image and the second image are respectively calculated according to the region luminance value divided by the region and the weight of the corresponding region The steps include:

   Calculating, respectively, a sum of a product of each of the region luminance values and the weights of the corresponding regions to obtain a total weight luminance value of the first image and the second image; and

   Calculating a ratio of the total weight luminance value to the total weight to obtain luminance values of the first image and the second image, respectively.
5. The difference calibration method according to claim 1, wherein said updating said exposure of said first camera and/or said second camera at a next moment when said image brightness difference value is greater than said predetermined threshold Step of the parameter include:

   Calculating a difference calibration parameter of the binocular vision system according to image brightness values and reference brightness values of the first image and the second image when the image brightness difference value is greater than the predetermined threshold; and

   Updating the exposure parameter of the first camera and/or the second camera at a next moment according to the difference calibration parameter and the current exposure parameter of the first camera or the second camera.
6. The difference calibration method according to claim 5, wherein the reference luminance value is an image luminance value of the first image, an image luminance value of the second image, or the first image and the first Any of the average image brightness values of the two images.
7. The difference calibration method according to claim 6, wherein the average image luminance value of the first image and the second image may be any one of a weighted average luminance value and an arithmetic average luminance value.
8. The difference calibration method according to claim 6, wherein said image brightness difference value comprises an absolute difference value, and said step of calculating an image brightness difference value between said first image and said second image comprises :

   And calculating the absolute difference value according to image brightness values of the first image and the second image and acquiring current difference calibration parameters corresponding to the first image and the second image, where the absolute difference value adopts the following condition Calculation:

   D=|a-(b/xb)*xa| or D=|(a/xa)*xb-b|;

   Wherein D is the absolute difference value, and a and b are image brightness values of the first image and the second image, respectively, and xa and xb are respectively corresponding to acquiring the first image and the second image. Current differential calibration parameters.
9. The difference calibration method according to claim 6, wherein said image luminance difference value includes a relative difference value, and said step of calculating an image luminance difference value between said first image and said second image includes :

   Calculating the relative difference value according to the image brightness value of the first image and the second image, the reference brightness value, and acquiring current difference calibration parameters corresponding to the first image and the second image, where The relative difference value is a ratio of the absolute difference value to the reference brightness value, and the absolute difference value is calculated by the following conditional expression:

   D=|a-(b/xb)*xa| or D=|(a/xa)*xb-b|;

   Where D is the absolute difference value, a and b are the brightness values of the first image and the second image, respectively, and xa and xb are respectively collected for the first image and the second image. Differential calibration parameters.
10. The difference calibration method according to claim 5, wherein said image brightness value and said reference brightness value of said first image and said second image are different when said image brightness difference value is greater than said predetermined threshold Calculate the The steps of the difference calibration parameters of the binocular vision system include:

    Calculating the first image and the image according to the image brightness value of the first image and the second image and the current difference calibration parameter corresponding to the first image and the second image and the reference brightness value The difference calibration parameter of the second image, the difference calibration parameter is calculated by the following conditional formula:

    a/xa*aedc=ref/xref*aedc_ref(1) and/or b/xb*aedc=ref/xref*aedc_ref(1);

    Where a and b are the image brightness values of the first image and the second image, respectively, xa and xb are respectively collecting current difference calibration parameters corresponding to the first image and the second image, and aedc is The difference calibration parameter, ref is the reference brightness value, xref is the difference calibration parameter corresponding to the reference brightness value, and aedc_ref(1) is the reference difference calibration parameter, where xref and aedc_ref(1) are both 1.
11. The difference calibration method according to claim 5, wherein the exposure parameter comprises an automatic exposure parameter and a calibration exposure parameter, the calibration parameter and the current state of the first camera or the second camera according to the difference Updating the exposure parameter of the next moment of the first camera and/or the second camera by the exposure parameter comprises:

    And calculating a calibration exposure parameter of the first camera and/or the second camera according to the difference calibration parameter and the current exposure parameter of the first camera and the second camera.
12. The difference calibration method according to claim 11, wherein the automatic exposure parameter comprises at least one of a coarse adjustment exposure time, an analog gain, and a digital gain, and the calibration exposure parameter includes a coarse adjustment exposure time and a fine adjustment exposure. At least one of time, line cycle time, analog gain, and digital gain.
13. The difference calibration method according to claim 12, wherein when the calculated calibration exposure parameter is different from the type of the automatic exposure parameter, the calibration parameter and the first camera and the In the step of calculating the calibration exposure parameter of the first camera and/or the second camera of the current exposure parameter of the second camera, the calibration exposure parameter is calculated by the following conditional expression:

    Y=r*c;

    Wherein Y is the calibration exposure parameter, r is a default exposure parameter of the first camera or the second camera, and c is the difference calibration parameter.
14. The difference calibration method according to claim 12, wherein when the calculated calibration exposure parameter and the automatic exposure parameter are all of the same type, the difference calibration parameter and the first camera and the In the step of calculating the calibration exposure parameter of the first camera and/or the second camera of the current exposure parameter of the second camera, the calibration exposure parameter is calculated by the following conditional expression:

    Y=g*x;

    Wherein Y is the calibration exposure parameter, g is an automatic exposure parameter corresponding to the first image and/or the second image, and c is the difference calibration parameter.
15. The difference calibration method according to claim 12, wherein said calculated calibration exposure parameter is identical to a portion of said automatic exposure parameter and the other portion is different, said calibration parameter and said said In the step of calculating the calibration exposure parameters of the first camera and/or the second camera by the current exposure parameter of a camera and the second camera, the calibration exposure parameter is calculated by the following conditional expression:

    Y1=(g*c1) and Y2=(r*c2);

    Wherein, Y1 and Y2 are the calibration exposure parameters, g is an automatic exposure parameter corresponding to the first image and/or the second image, and r is a default of the first camera or the second camera. The exposure parameter, c1*c2=c and c is the difference calibration parameter.
16. A binocular vision system, comprising: a first camera, a second camera and a processor, the processor is configured to:

    Acquiring image brightness values of the first image and the second image of the same scene currently acquired by the first camera and the second camera in real time;

    Calculating an image brightness difference value between the first image and the second image;

    Determining whether the image brightness difference value is greater than a predetermined threshold; and

    And updating an exposure parameter of the first camera and/or the second camera at a next moment when the image brightness difference value is greater than the predetermined threshold.
17. The binocular vision system of claim 16 wherein said processor is operative to:

    Obtaining a region division of the first image and the second image and a weight of the corresponding region; and

    And calculating image brightness values of the first image and the second image according to the area brightness value of the area and the weight of the corresponding area.
18. The binocular vision system of claim 17 wherein said processor is operative to:

    Obtaining an operating state of the binocular vision system;

    Determining an area of interest of the first image and the second image according to the working state; and

    Acquiring the weight of the region division and the corresponding region of the first image and the second image according to the region of interest.
19. The binocular vision system of claim 18 wherein said processor is operative to:

    Calculating, respectively, a sum of a product of each of the region luminance values and the weights of the corresponding regions to obtain a total weight luminance value of the first image and the second image; and

    Calculating a ratio of the total weight luminance value to the total weight to obtain luminance values of the first image and the second image, respectively.
20. The binocular vision system of claim 16 wherein said processor is operative to:

    Calculating a difference calibration parameter of the binocular vision system according to image brightness values and reference brightness values of the first image and the second image when the image brightness difference value is greater than the predetermined threshold; and

    Updating the exposure parameter of the first camera and/or the second camera at a next moment according to the difference calibration parameter and the current exposure parameter of the first camera or the second camera.
21. The binocular vision system of claim 20 wherein said image brightness difference value comprises an absolute difference value, said processor for:

    And calculating the absolute difference value according to image brightness values of the first image and the second image and acquiring current difference calibration parameters corresponding to the first image and the second image, where the absolute difference value adopts the following condition Calculation:

    D=|a-(b/xb)*xa| or D=|(a/xa)*xb-b|;

    Wherein D is the absolute difference value, and a and b are image brightness values of the first image and the second image, respectively, and xa and xb are respectively corresponding to acquiring the first image and the second image. Current differential calibration parameters.
22. The binocular vision system of claim 20 wherein said image brightness difference value comprises a relative difference value, said processor for:

    Calculating the relative difference value according to the image brightness value of the first image and the second image, the reference brightness value, and acquiring current difference calibration parameters corresponding to the first image and the second image, where The relative difference value is a ratio of the absolute difference value to the reference brightness value, and the absolute difference value is calculated by the following conditional expression:

    D=|a-(b/xb)*xa| or D=|(a/xa)*xb-b|;

    Where D is the absolute difference value, a and b are the brightness values of the first image and the second image, respectively, and xa and xb are respectively collected for the first image and the second image. Differential calibration parameters.
23. The binocular vision system of claim 20 wherein said processor is operative to:

    Calculating the first image and the image according to the image brightness value of the first image and the second image and the current difference calibration parameter corresponding to the first image and the second image and the reference brightness value The second image The difference calibration parameters are calculated, and the difference calibration parameters are calculated by the following conditional formula:

    a/xa*aedc=ref/xref*aedc_ref(1) and/or b/xb*aedc=ref/xref*aedc_ref(1);

    Where a and b are the image brightness values of the first image and the second image, respectively, xa and xb are respectively collecting current difference calibration parameters corresponding to the first image and the second image, and aedc is The difference calibration parameter, ref is the reference brightness value, xref is the difference calibration parameter corresponding to the reference brightness value, and aedc_ref(1) is the reference difference calibration parameter, where xref and aedc_ref(1) are both 1.
24. The binocular vision system of claim 20 wherein said exposure parameters comprise automatic exposure parameters and calibration exposure parameters, said processor for:

    And calculating a calibration exposure parameter of the first camera and/or the second camera according to the difference calibration parameter and the current exposure parameter of the first camera and the second camera.
25. A computer readable storage medium comprising a computer program for use in conjunction with a binocular vision system, the computer program being executable by a processor to perform the difference calibration method of any of claims 1-15.

Images