CN116524041A - Camera calibration method, device, equipment and medium - Google Patents

Abstract

The embodiment of the specification discloses a camera calibration method, device, equipment and medium. Comprising the following steps: and acquiring a calibration image acquired by the camera aiming at the calibration object. Wherein, the target two-dimensional code pattern is drawn in the calibration object. And carrying out two-dimensional code identification processing on the calibration image to obtain a first pixel coordinate set aiming at each target vertex under a preset pixel coordinate system. And determining world coordinates of all target vertexes on the calibration object by using a preset world coordinate system to obtain a world coordinate set. And completing the calibration of the camera according to the first pixel coordinate set and the world coordinate set. The scheme can simplify the camera calibration flow, is convenient for realizing the camera calibration of the industrial assembly line, and can improve the simplicity of calibrating the cameras in batches.

Description

A camera calibration method, device, equipment and medium

technical field

The present application relates to the technical field of camera calibration, and in particular to a camera calibration method, device, equipment and medium.

Background technique

In the existing camera calibration methods, the calibration images used need to contain calibration patterns with preset rules. During the process of collecting calibration images for the calibration objects, due to the influence of factors such as the brightness of the shooting light and the angle of the calibration objects, there will inevitably be calibration images that do not meet the preset requirements in the captured calibration images. Therefore, in order to obtain qualified calibration images, when collecting calibration images for calibration objects, it is necessary to collect multiple calibration images in each camera pose state, and then select from the multiple calibration images taken by manual screening. A qualified calibration image is produced. If the calibration images taken are not screened, poorly imaged samples may be introduced. Therefore, the camera calibration process in the existing camera calibration method takes a long time, the process is cumbersome, it is difficult to realize the industrialized assembly line operation, and it is more difficult to calibrate the cameras in batches.

Based on this, how to simplify the camera calibration process, realize the industrialized assembly line operation of camera calibration, and improve the simplicity of batch calibration of cameras has become an urgent technical problem to be solved.

Contents of the invention

The embodiment of this specification provides a camera calibration method, device, equipment, and medium to solve the problem that the process of calibrating cameras in the prior art takes a long time, the process is cumbersome, and it is difficult to realize industrialized assembly line operations, and batch calibration of cameras difficult technical problems.

In order to solve the above-mentioned technical problems, the embodiments of this specification are implemented as follows:

A camera calibration method provided in an embodiment of this specification includes:

The calibration image collected by the camera for the calibration object is acquired, and the target two-dimensional code pattern is drawn in the calibration object.

Using a two-dimensional code recognition algorithm, the calibration image is processed to obtain a first set of pixel coordinates in a preset pixel coordinate system; the first set of pixel coordinates includes the identified from the calibration image The first pixel coordinate of the target vertex of the target QR code pattern.

Determine the world coordinates of each target vertex in the preset world coordinate system to obtain a world coordinate set.

A camera calibration algorithm is used to perform camera calibration processing according to the first set of pixel coordinates and the set of world coordinates to obtain a calibration result of the camera.

Preferably, the calibration image includes a first calibration image and a second calibration image collected for a calibration object at any two different poses, the calibration object includes a first calibration plate and a second calibration plate, and the first The target two-dimensional code pattern is drawn on both the calibration plate and the second calibration plate, and there is a preset angle between the first calibration plate and the second calibration plate.

Preferably, the camera calibration algorithm is used to perform camera calibration processing according to the first pixel coordinate set and the world coordinate set to obtain the camera calibration result, including:

Using the conversion relationship between the preset world coordinate system and the preset pixel coordinate system, determine the world coordinate of the target vertex in the world coordinate set corresponding to the first pixel coordinate system under the preset pixel coordinate system Two pixel coordinates to get the second set of pixel coordinates.

Using a camera calibration algorithm, perform calibration processing on the camera according to the first set of pixel coordinates and the second set of pixel coordinates to obtain a calibration result of the camera.

Preferably, performing calibration processing on the camera according to the first set of pixel coordinates and the second set of pixel coordinates to obtain a calibration result of the camera includes:

Determine at least one target pixel coordinate set from the first pixel coordinate set and the second pixel coordinate set; each pixel coordinate in each target pixel coordinate set belongs to the same target vertex, and The set of target pixel coordinates includes one of the first pixel coordinates and one of the second pixel coordinates.

According to each set of target pixel coordinates, a least squares problem is constructed.

Calibrate the camera according to the least square problem to obtain a calibration result of the camera.

Preferably, determining at least one target pixel coordinate set from the first pixel coordinate set and the second pixel coordinate set includes:

For any one of the first pixel coordinates in the first pixel coordinate set, determine the positioning information of the first pixel coordinates; the positioning information includes: the target two-dimensional code to which the first pixel coordinates belong The decoding information, the position information of the target vertex corresponding to the first pixel coordinate in the target two-dimensional code, and the target marking plate to which the first pixel coordinate belongs, the target marking plate is the The first calibration plate or the second calibration plate.

The second pixel coordinates matching the first pixel coordinates are determined from the second pixel coordinate set according to the positioning information of the first pixel coordinates.

A set of target pixel coordinates is generated according to the first pixel coordinates and the second pixel coordinates matching the first pixel coordinates.

Preferably, the camera is calibrated according to the least squares problem to obtain a calibration result of the camera, including:

The nonlinear optimization algorithm is used to perform iterative optimization processing on the least squares problem to obtain the calculation result of the least squares problem.

Judging whether the calculation result of the least squares problem is less than or equal to a preset threshold value to obtain a judgment result.

If the judgment result indicates that the calculation result of the least squares problem is less than or equal to a preset threshold, then the camera parameters corresponding to the calculation result are determined as the calibration result of the camera.

Preferably, the calibration result of the camera includes the calibration result of the camera intrinsic parameters and the calibration result of the camera distortion coefficient.

The calculation formula of the least squares problem is:

Among them: argmin represents the calculated minimum value, Ui represents the first pixel coordinates, Pi represents the world coordinates of the target vertex in the preset world coordinate system, K represents the participation of the camera in the camera Distortion coefficient, T represents the camera extrinsics of the camera, KTPi represents the second pixel coordinates, and i represents the number of target vertices.

A camera calibration device, comprising,

The first acquisition module is configured to acquire the calibration image collected by the camera for the calibration object, and the target two-dimensional code pattern is drawn in the calibration object.

A processing module, configured to use a two-dimensional code recognition algorithm to process the calibration image to obtain a first set of pixel coordinates in a preset pixel coordinate system; the first set of pixel coordinates includes The first pixel coordinates of the identified target vertex of the target two-dimensional code pattern.

The determining module is configured to determine the world coordinates of each target vertex in the preset world coordinate system to obtain a set of world coordinates.

The calibration module is configured to use a camera calibration algorithm to perform camera calibration processing according to the first set of pixel coordinates and the set of world coordinates to obtain a calibration result of the camera.

A camera calibration device includes a memory, a processor and a computer program stored on the memory, and the processor executes the computer program to implement the above method.

A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above method can be realized.

At least one embodiment provided in this specification can achieve the following beneficial effects:

This solution uses the target vertices detected and identified from the two-dimensional code pattern in the calibration image, and under a specific calibration algorithm, the camera calibration process can be completed. So that the calibration system can automatically complete the calibration of the camera after receiving the calibration image, without manual participation in the selection of qualified calibration images, so as to simplify the process of camera calibration, reduce the time-consuming camera calibration, and facilitate the realization of camera calibration Industrialized assembly line operations to improve the simplicity of batch calibration of cameras.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments described in this application. Those skilled in the art can also obtain other drawings based on these drawings without any creative effort.

FIG. 1 is a schematic flow diagram of a camera calibration method provided by an embodiment of this specification;

Fig. 2 is a schematic diagram of a two-dimensional code pattern provided by the embodiment of this specification;

Fig. 3 is a schematic diagram of the projected position of a spatial position point P on the imaging plane and the observed pixel coordinate position provided by the embodiment of this specification;

Fig. 4 is a schematic diagram of a preset world coordinate system provided by the embodiment of this specification;

Fig. 5 is a schematic diagram of the calibration pattern of the first calibration plate provided by the embodiment of this specification;

Fig. 6 is a schematic diagram of the calibration pattern of the second calibration plate provided by the embodiment of this specification;

Fig. 7 is a schematic structural diagram of a camera calibration device corresponding to Fig. 1 provided by the embodiment of this specification;

FIG. 8 is a schematic structural diagram of a camera calibration device corresponding to FIG. 1 provided by the embodiment of this specification.

Detailed ways

In order to make the purpose, technical solutions and advantages of one or more embodiments of this specification more clear, the following will clearly and completely describe the technical solutions of one or more embodiments of this specification in conjunction with specific embodiments of this specification and corresponding drawings . Apparently, the described embodiments are only some of the embodiments in this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of one or more embodiments in this specification.

The technical solutions provided by various embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

FIG. 1 is a schematic flowchart of a camera calibration method provided by an embodiment of this specification. From a program point of view, the subject of execution of the process may be a device for calibrating the camera, or an application program carried on the device for calibrating the camera. As shown in Figure 1, the process may include the following steps:

Step 102: Obtain a calibration image captured by the camera for the calibration object, in which the target two-dimensional code pattern is drawn.

In the embodiment of this specification, the camera calibration object may be a calibration device for displaying a calibration image required for calibrating the camera. The calibration pattern drawn on the calibration object can be a pattern composed of a first preset number of two-dimensional codes. The first preset number can be any natural number greater than or equal to 1. The larger the first preset number, the camera is calibrated The higher the accuracy of the result. When the first preset number is any natural number greater than 1, the two-dimensional codes on the calibration object can be all the same two-dimensional codes, or partly the same two-dimensional codes, or can be different from each other. the same QR code. The type of the two-dimensional code can be tag36h11 type or other types of two-dimensional code. Each QR code in the camera calibration object is labeled differently. The target two-dimensional code pattern may be one or more two-dimensional code patterns drawn on the calibration object.

In the embodiment of this specification, in the process of calibrating the camera, the number of calibration images can be any number greater than or equal to 1. The larger the number of calibration images, the higher the accuracy of the calibration result of the camera.

Step 104: Process the calibration image using a two-dimensional code recognition algorithm to obtain a first set of pixel coordinates in a preset pixel coordinate system; the first set of pixel coordinates includes The first pixel coordinates of the target vertex of the target two-dimensional code pattern.

In the embodiment of this specification, as shown in FIG. 2 , any two-dimensional code pattern selected from the calibration image includes four vertices from 0 to 4. The sequence numbers of the vertices can be set according to requirements, and the vertices included in the two-dimensional code pattern can also be other vertices identified from the two-dimensional code pattern. Using a two-dimensional code recognition algorithm to obtain the pixel coordinates of any vertex included in the two-dimensional code in the corresponding calibration image. The QR code recognition algorithm can be to call the extractTags function in the opencv open source library. The two-dimensional code recognition algorithm may also be other algorithms for obtaining the pixel coordinates at the vertices of the two-dimensional code from the calibration image. Based on the same principle, the vertex coordinates of other two-dimensional code patterns on the calibration image can be obtained. The number of target vertices can be one or more. When there is one target vertex, the target vertex can be any vertex identified for any two-dimensional code pattern in the calibration object; when there are multiple target vertices, the target vertex can be is a plurality of vertices identified for any one or more two-dimensional code patterns in the calibration object.

Step 106: Determine the world coordinates of each target vertex in the preset world coordinate system to obtain a world coordinate set.

In the embodiment of this specification, the preset world coordinate system may be a world coordinate system composed of the upper left corner of the calibration object as the origin, the X-axis is horizontally left, the Y-axis is horizontally downward, and the Z-axis is 0. Under the preset world coordinate system, obtain the world coordinates at the vertices of each two-dimensional code pattern on the calibration object, and obtain the world coordinate set. It should be noted that the preset world coordinate system can also be selected according to other principles, and this scheme does not specifically limit the selection of the preset world coordinate system.

Step 108: Using a camera calibration algorithm, perform camera calibration processing according to the first pixel coordinate set and the world coordinate set, and obtain a calibration result of the camera.

In the embodiment of this specification, as shown in Figure 3, the principle of the camera calibration algorithm used in the present invention can be to obtain any point P in the spatial position, P1 represents the projected position of point P, and P2 represents the observed point P By adjusting the parameters of the camera, the projected position of point P is gradually approached to the observed pixel position. When the distance between the projected position of point P and the observed pixel position is less than the preset threshold, stop Adjust the camera parameters, and the camera parameters obtained at this time are the calibration results after the camera is calibrated. Any world coordinate in the world coordinate set can be used as the spatial position of point P, then the pixel position observed for point P can be obtained from the first pixel coordinate set, and the projection of point P can be generated according to the world coordinate of point P Coordinates, therefore, based on the above principles, the first pixel coordinate set and the world coordinate set can be used to complete the calibration process for the camera.

In the embodiment of this specification, after receiving the calibration image, the solution can automatically complete the calibration of the camera under a specific calibration algorithm without manual participation in screening qualified calibration images, so as to simplify the process of camera calibration and reduce the number of cameras. Calibration is time-consuming, and it is convenient to realize industrial assembly line operation for camera calibration, so as to improve the simplicity of batch calibration for cameras.

Based on the method in FIG. 1 , the embodiment of this specification also provides some specific implementations of the method, which will be described below.

The calibration image includes a first calibration image and a second calibration image collected for a calibration object at any two different poses, the calibration object includes a first calibration board and a second calibration board, and the first calibration board and The target two-dimensional code pattern is drawn on each of the second calibration boards, and there is a preset angle between the first calibration board and the second calibration board.

In the embodiment of this specification, the camera calibration object may include a first calibration plate and a second calibration plate, and the first calibration plate and the second calibration plate may be connected together through a common edge, or they may not be connected together Preset distance apart. There may be a preset angle between the first calibration plate and the second calibration plate, and the preset angle may be 90 degrees, or any angle greater than 0 degrees and less than or equal to 180 degrees. The shapes of the first calibration plate and the second calibration plate can be any one of square, rectangle, circle and other irregular figures, and the size and shape of the first calibration plate and the second calibration plate can be the same or different. same. The size and shape of the first calibration plate and the second calibration plate are not specifically limited.

It should be noted that when the angle between the first calibration plate and the second calibration plate is 90 degrees, the preset world coordinate system can be set in the manner shown in FIG. 4 . As shown in Figure 4, take the straight line parallel to the bottom edge of the first calibration plate on the ground as the X coordinate axis, take the straight line parallel to the bottom edge of the second calibration plate on the ground as the Y coordinate axis, and the X coordinate axis and the Y coordinate axis The intersection point of the axes is the coordinate origin O, the straight line passing through the coordinate origin O perpendicular to the ground is the Z coordinate axis, and the coordinate system composed of the X coordinate axis, the Y coordinate axis, the Z coordinate axis and the coordinate origin O is the preset world coordinate system. With such an arrangement, it is convenient to obtain the world coordinates of each target vertex on the first calibration board and the second calibration board.

In the embodiment of this specification, a pattern composed of a second preset number of two-dimensional codes can be drawn on the first calibration plate, and the second preset number can be any natural number greater than or equal to 1. When the second preset number is When any natural number is greater than 1, the two-dimensional codes on the first calibration board can be all the same two-dimensional codes, or partly the same two-dimensional codes, or can be different two-dimensional codes , the two-dimensional codes contained on the first calibration board may be arranged in n rows and n columns, or in other ways. The calibration pattern drawn on the second calibration plate is consistent with the calibration pattern drawn on the first calibration plate. It should be noted that the consistency of the calibration pattern drawn on the first calibration plate and the second calibration means that the area shape of the calibration pattern on the first calibration plate is consistent with the area shape of the calibration pattern on the second calibration, and the first After the calibration pattern in the calibration plate is overlapped with the calibration pattern in the second calibration plate, the size and shape of the two-dimensional code at the overlapping position are consistent. As shown in Figure 5, the calibration pattern of the first calibration board includes a two-dimensional code pattern with 6 rows and 6 columns, and as shown in Figure 6, the calibration pattern of the second calibration board also includes a two-dimensional code pattern with 6 rows and 6 columns .

In the embodiment of this specification, the first calibration image may be any one or more calibration images captured by the camera in the first posture state for the calibration pattern. The first pose state can be the state when there is a preset distance between the optical center of the camera and the center of the calibration pattern. The preset distance can be 60 cm, or other distances can be selected according to actual needs. The first pose state The camera lens needs to be aimed at the exact center of the calibration pattern. The second calibration pattern may be any one or more calibration images taken with respect to the calibration pattern in the second pose state of the camera. The second pose state may be a state in which the camera lens is rotated by a preset angle based on the first pose state, and the preset included angle may be 90 degrees or other rotation angles. The first pose state and the second pose state are different camera poses. Both the first calibration image and the second calibration image may contain the calibration pattern on the first calibration plate and the calibration pattern on the second calibration plate.

Step 108: Using the camera calibration algorithm to perform camera calibration processing according to the first pixel coordinate set and the world coordinate set to obtain the camera calibration result may include:

Using the conversion relationship between the preset world coordinate system and the preset pixel coordinate system, determine the world coordinate of the target vertex in the world coordinate set corresponding to the first pixel coordinate system under the preset pixel coordinate system Two pixel coordinates to get the second set of pixel coordinates.

Using a camera calibration algorithm, perform calibration processing on the camera according to the first set of pixel coordinates and the second set of pixel coordinates to obtain a calibration result of the camera.

In the embodiment of this specification, the conversion relationship between the preset world coordinate system and the preset pixel coordinate system can be

ZP _uv =KTP _W , where: P _uv represents pixel coordinates, P _W represents world coordinates, Z represents a scale factor, K represents internal camera parameters, and T represents external camera parameters. According to the conversion relationship, the world coordinates of each target vertex in the world coordinate set are converted into the second pixel coordinates to obtain the second pixel coordinate set. It should be noted that the second pixel coordinates may be used as the projected position coordinates of the target vertex. Both the first pixel coordinates and the second pixel coordinates of the target vertex are located on the same imaging plane, therefore, both the first pixel coordinates and the second pixel coordinates may be pixel coordinates in a preset pixel coordinate system. Under the principle of the above camera calibration algorithm, by adjusting the parameters of the camera so that the distance between the second pixel coordinate and the first pixel coordinate of the target vertex is less than the preset threshold, the obtained camera parameters are the camera calibration After calibration results.

In the process of calibrating the camera, in order to improve the accuracy of the calibration result of the camera, it is necessary to calibrate the camera according to multiple target vertices. Based on this, the performing calibration processing on the camera according to the first set of pixel coordinates and the second set of pixel coordinates to obtain a calibration result of the camera may include:

Determine at least one target pixel coordinate set from the first pixel coordinate set and the second pixel coordinate set; each pixel coordinate in each target pixel coordinate set belongs to the same target vertex, and The set of target pixel coordinates includes one of the first pixel coordinates and one of the second pixel coordinates.

According to each set of target pixel coordinates, a least squares problem is constructed.

Calibrate the camera according to the least square problem to obtain a calibration result of the camera.

In the embodiment of this specification, any first pixel coordinate x ₁ is selected from the first pixel coordinate set, and the target vertex to which the first pixel coordinate belongs is determined. Assuming that the target vertex is vertex a, then in the second pixel coordinate set The second pixel coordinate y ₁ of the vertex a is determined in , then the first pixel coordinate x ₁ and the second pixel coordinate y ₁ constitute a set of target pixel coordinates. According to the above method, a plurality of target pixel coordinate sets are sequentially determined, any number of target pixel coordinate sets are selected from the determined multiple target pixel coordinate sets to construct a least squares problem, and the constructed least squares problem is performed Solving, when the result of the least squares problem meets the preset requirements, the corresponding camera parameters are the calibration results after the camera is calibrated. It should be noted that when constructing the least squares problem, there is no specific limit on the number of target pixel coordinate sets. The more the number, the higher the accuracy of the calibration results after the camera is calibrated. The accuracy of the final calibration result is lower.

The determining at least one target pixel coordinate set from the first pixel coordinate set and the second pixel coordinate set may include:

For any one of the first pixel coordinates in the first pixel coordinate set, determine the positioning information of the first pixel coordinates; the positioning information includes: the target two-dimensional code to which the first pixel coordinates belong The decoding information, the position information of the target vertex corresponding to the first pixel coordinate in the target two-dimensional code, and the target marking plate to which the first pixel coordinate belongs, the target marking plate is the The first calibration plate or the second calibration plate.

The second pixel coordinates matching the first pixel coordinates are determined from the second pixel coordinate set according to the positioning information of the first pixel coordinates.

A set of target pixel coordinates is generated according to the first pixel coordinates and the second pixel coordinates matching the first pixel coordinates.

In the embodiment of this specification, after selecting the first pixel coordinate x ₁ from the first pixel coordinate set, the pixel coordinate information of the first pixel coordinate x ₁ is analyzed to obtain the target vertex a to which the first pixel coordinate x ₁ belongs. Position information, according to the position information of the target vertex a, obtain the position information of the target two-dimensional code b to which the target vertex a belongs, analyze the target two-dimensional code b, determine the target calibration board to which the target two-dimensional code b belongs, and target calibration The board belongs to the first calibration board or the second calibration board. Then, according to the target calibration plate, the position information of the target two-dimensional code b, and the position information of the target vertex a, the second pixel coordinate y ₁ matching it is determined from the second pixel coordinate set, and the second pixel coordinate y ₁ is the The world coordinates of the target vertex a are transformed into pixel coordinates by the conversion relationship between the world coordinate system and the pixel coordinate system. After the first pixel coordinate _x1 and the first pixel coordinate _y1 are obtained, a target pixel coordinate set is generated according to the first pixel coordinate _x1 and the first pixel coordinate _y1 .

The step of performing calibration processing on the camera according to the least squares problem to obtain a calibration result of the camera may include:

The nonlinear optimization algorithm is used to perform iterative optimization processing on the least squares problem to obtain the calculation result of the least squares problem.

Judging whether the calculation result of the least squares problem is less than or equal to a preset threshold value to obtain a judgment result.

If the judgment result indicates that the calculation result of the least squares problem is less than or equal to a preset threshold, then the camera parameters corresponding to the calculation result are determined as the calibration result of the camera.

The calibration result of the camera includes the calibration result of the internal camera parameters and the calibration result of the camera distortion coefficient of the camera.

The calculation formula of the least squares problem is:

Among them: argmin represents the calculated minimum value, Ui represents the first pixel coordinates, Pi represents the world coordinates of the target vertex in the preset world coordinate system, K represents the participation of the camera in the camera Distortion coefficient, T represents the camera extrinsics of the camera, KTPi represents the second pixel coordinates, and i represents the number of target vertices.

In the embodiment of this specification, according to any set of target pixel coordinates, a distance difference between the first pixel coordinates and the second pixel coordinates is generated, and a least squares problem is generated by using multiple distance differences between the first pixel coordinates and the second pixel coordinates The calculation formula The nonlinear optimization algorithm can be used to iteratively optimize the least squares problem. The specific iterative steps can be:

Step 1: Set the initial values of the camera parameters and start the optimization process. The horizontal focal length f _x of the camera can be set to the real focal length/pixel size of the camera, the vertical focal length f _y of the camera can be set to the real focal length/pixel size of the camera, and the optical center c _x of the camera can be set to the width/2 of the image , the optical center c _y of the camera can be set to the image height/2.

Step 2: Adjust the camera parameters to obtain the current calculation result of the least squares problem. The adjustment of the camera parameters can increase a preset value for each iteration of the camera parameters, and the preset value can be set according to actual needs.

Step 3: Determine whether the current calculation result is less than or equal to the preset threshold, if the current calculation result is greater than the preset threshold, return to step 2; if the current calculation result is less than or equal to the preset threshold, stop the iterative process. The preset threshold can be set according to the requirements for the accuracy of the camera calibration results.

After optimizing and iteratively processing the constructed least squares problem to obtain the required calculation results, obtain the corresponding camera parameters at this time. The camera parameters are the calibration results after the camera is calibrated. The camera parameters can include the camera intrinsic parameter f _x ,f _y ,c _x ,c _y , camera radial distortion parameters k ₁ ,k ₂ ,k ₃ , and camera tangential distortion parameters p ₁ ,p ₂ . Among them, the nonlinear optimization algorithm used can be the Levenberg-Marquardt method, or other algorithms that can solve the least squares problem. This scheme does not specifically limit the nonlinear optimization algorithm .

In the embodiment of this specification, by constructing the least squares problem, it can be realized that the distances between the observed pixel coordinate positions and the projected positions of multiple target vertices all meet the preset requirements. Therefore, the parameters of the camera can be obtained on the premise that the distances between the observed pixel coordinate positions and the projection positions of multiple target vertices meet the preset requirements, so as to improve the accuracy of calibration of the camera parameters.

Based on the same idea, the embodiment of this specification also provides a device corresponding to the above method. FIG. 7 is a schematic structural diagram of a camera calibration device corresponding to FIG. 1 provided by the embodiment of this specification. As shown in Figure 7, the device may include:

The acquiring module 702 is configured to acquire a calibration image captured by the camera for the calibration object, where the target two-dimensional code pattern is drawn.

The processing module 704 is configured to use a two-dimensional code recognition algorithm to process the calibration image to obtain a first set of pixel coordinates in a preset pixel coordinate system; the first set of pixel coordinates includes The first pixel coordinates of the target vertex of the target two-dimensional code pattern identified in .

The determination module 706 is configured to determine the world coordinates of each target vertex in the preset world coordinate system to obtain a world coordinate set.

The calibration module 708 is configured to use a camera calibration algorithm to perform camera calibration processing according to the first set of pixel coordinates and the set of world coordinates to obtain a calibration result of the camera.

Optionally, the calibration image includes a first calibration image and a second calibration image collected for a calibration object at any two different poses, the calibration object includes a first calibration board and a second calibration board, and the first calibration The target two-dimensional code pattern is drawn on both the first calibration board and the second calibration board, and there is a preset angle between the first calibration board and the second calibration board.

Optionally, the calibration module 708 may include:

The first determining unit is configured to use the conversion relationship between the preset world coordinate system and the preset pixel coordinate system to determine the world coordinate of the target vertex in the world coordinate set corresponding to the preset Set the second pixel coordinates in the pixel coordinate system to obtain a second set of pixel coordinates.

The calibration unit is configured to use a camera calibration algorithm to perform calibration processing on the camera according to the first set of pixel coordinates and the second set of pixel coordinates to obtain a calibration result of the camera.

Optionally, the calibration unit may include:

The first determining subunit is configured to determine at least one target pixel coordinate set from the first pixel coordinate set and the second pixel coordinate set; each pixel coordinate in each of the target pixel coordinate sets belongs to At the same target vertex, and the set of target pixel coordinates includes one of the first pixel coordinates and one of the second pixel coordinates.

A subunit is constructed, configured to construct a least squares problem according to each set of target pixel coordinates.

The calibration subunit is configured to perform calibration processing on the camera according to the least square problem to obtain a calibration result of the camera.

Optionally, the first determining subunit may include:

The first determining subunit is specifically configured to determine the positioning information of the first pixel coordinates for any one of the first pixel coordinates in the first pixel coordinate set; the positioning information includes: the first pixel coordinates The decoding information of the target two-dimensional code, the position information of the target vertex corresponding to the first pixel coordinate in the target two-dimensional code, and the target calibration plate to which the first pixel coordinate belongs , the target calibration plate is the first calibration plate or the second calibration plate.

The first determination subunit is specifically configured to determine the second pixel coordinates matching the first pixel coordinates from the second pixel coordinate set according to the positioning information of the first pixel coordinates.

The first determining subunit is specifically configured to generate a set of target pixel coordinates according to the first pixel coordinates and the second pixel coordinates matching the first pixel coordinates.

Optionally, the calibration subunit may include:

The calibration subunit is specifically configured to use a nonlinear optimization algorithm to iteratively optimize the least squares problem to obtain a calculation result of the least squares problem.

The calibration subunit is specifically used to judge whether the calculation result of the least squares problem is less than or equal to a preset threshold value, and obtain the judgment result.

The calibration subunit is specifically configured to determine the camera parameter corresponding to the calculation result as the calibration result of the camera if the judgment result indicates that the calculation result of the least squares problem is less than or equal to a preset threshold.

Optionally, the calibration result of the camera includes a calibration result of internal camera parameters and a calibration result of a camera distortion coefficient of the camera.

The calculation formula of the least squares problem is:

Among them: argmin represents the calculated minimum value, Ui represents the first pixel coordinates, Pi represents the world coordinates of the target vertex in the preset world coordinate system, K represents the participation of the camera in the camera Distortion coefficient, T represents the camera extrinsics of the camera, KTPi represents the second pixel coordinates, and i represents the number of target vertices.

Based on the same idea, the embodiment of this specification also provides a device corresponding to the above method.

FIG. 8 is a schematic structural diagram of a camera calibration device corresponding to FIG. 1 provided by the embodiment of this specification. As shown in FIG. 8, device 800 may include:

at least one processor 810; and,

a memory 830 communicatively coupled to the at least one processor; wherein,

The memory 830 stores instructions 820 executable by the at least one processor 810, the instructions are executed by the at least one processor 810, so that the at least one processor 810 can execute the instructions to achieve the above-mentioned method.

The present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method can be realized.

It should be understood that in the methods described in one or more embodiments of this specification, the order of some steps may be adjusted according to actual needs, or some steps may be omitted.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and the computer-readable storage medium shown in FIG. 8 , since they are basically similar to the method embodiments, the description is relatively simple, and for relevant parts, refer to the part of the description of the method embodiments.

The above descriptions are only examples of the present application, and are not intended to limit the present application. For those skilled in the art, various modifications and changes may occur in this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims (10)

1. A camera calibration method, characterized in that, comprising: Obtain the calibration image collected by the camera for the calibration object, in which the target two-dimensional code pattern is drawn; Using a two-dimensional code recognition algorithm, the calibration image is processed to obtain a first set of pixel coordinates in a preset pixel coordinate system; the first set of pixel coordinates includes the identified from the calibration image The first pixel coordinate of the target vertex of the target two-dimensional code pattern; Determining the world coordinates of each target vertex in the preset world coordinate system to obtain a set of world coordinates; A camera calibration algorithm is used to perform camera calibration processing according to the first set of pixel coordinates and the set of world coordinates to obtain a calibration result of the camera. 2. The method according to claim 1, wherein the calibration image comprises a first calibration image and a second calibration image collected for a calibration object under any two different poses, and the calibration object includes the first A calibration plate and a second calibration plate, the target two-dimensional code pattern is drawn in the first calibration plate and the second calibration plate, and there is a predetermined gap between the first calibration plate and the second calibration plate. Set angle. 3. The method according to claim 2, wherein the camera calibration algorithm is used to perform camera calibration processing according to the first pixel coordinate set and the world coordinate set to obtain a calibration result of the camera, including : Using the conversion relationship between the preset world coordinate system and the preset pixel coordinate system, determine the world coordinate of the target vertex in the world coordinate set corresponding to the first pixel coordinate system under the preset pixel coordinate system Two pixel coordinates to obtain the second set of pixel coordinates; Using a camera calibration algorithm, perform calibration processing on the camera according to the first set of pixel coordinates and the second set of pixel coordinates to obtain a calibration result of the camera. 4. The method according to claim 3, wherein the calibration process is performed on the camera according to the first set of pixel coordinates and the second set of pixel coordinates to obtain a calibration result of the camera, including : Determine at least one target pixel coordinate set from the first pixel coordinate set and the second pixel coordinate set; each pixel coordinate in each target pixel coordinate set belongs to the same target vertex, and The set of target pixel coordinates includes one of the first pixel coordinates and one of the second pixel coordinates; Constructing a least squares problem according to each set of target pixel coordinates; Calibrate the camera according to the least square problem to obtain a calibration result of the camera. 5. The method according to claim 4, wherein the determining at least one target pixel coordinate set from the first pixel coordinate set and the second pixel coordinate set comprises: For any one of the first pixel coordinates in the first pixel coordinate set, determine the positioning information of the first pixel coordinates; the positioning information includes: the target two-dimensional code to which the first pixel coordinates belong The decoding information, the position information of the target vertex corresponding to the first pixel coordinate in the target two-dimensional code, and the target marking plate to which the first pixel coordinate belongs, the target marking plate is the The first calibration plate or the second calibration plate; determining the second pixel coordinates matching the first pixel coordinates from the second pixel coordinate set according to the positioning information of the first pixel coordinates; A set of target pixel coordinates is generated according to the first pixel coordinates and the second pixel coordinates matching the first pixel coordinates. 6. The method according to claim 4, wherein said calibration process is performed on said camera according to said least squares problem to obtain a calibration result of said camera, comprising: performing iterative optimization processing on the least squares problem by using a nonlinear optimization algorithm to obtain a calculation result of the least squares problem; Judging whether the calculation result of the least squares problem is less than or equal to a preset threshold value, and obtaining the judgment result; If the judgment result indicates that the calculation result of the least squares problem is less than or equal to a preset threshold, then the camera parameters corresponding to the calculation result are determined as the calibration result of the camera. 7. The method according to claim 6, wherein the calibration result of the camera comprises a calibration result of the internal camera parameters and a calibration result of the camera distortion coefficient of the camera; The calculation formula of the least squares problem is: Among them: argmin represents the calculated minimum value, Ui represents the first pixel coordinates, Pi represents the world coordinates of the target vertex in the preset world coordinate system, K represents the participation of the camera in the camera Distortion coefficient, T represents the camera extrinsics of the camera, KTPi represents the second pixel coordinates, and i represents the number of target vertices. 8. A camera calibration device, characterized in that, comprising: An acquisition module, configured to acquire a calibration image collected by the camera for a calibration object, wherein a target two-dimensional code pattern is drawn in the calibration object; A processing module, configured to use a two-dimensional code recognition algorithm to process the calibration image to obtain a first set of pixel coordinates in a preset pixel coordinate system; the first set of pixel coordinates includes The identified first pixel coordinates of the target vertex of the target two-dimensional code pattern; A determining module, configured to determine the world coordinates of each target vertex in the preset world coordinate system, to obtain a world coordinate set; The calibration module is configured to use a camera calibration algorithm to perform camera calibration processing according to the first set of pixel coordinates and the set of world coordinates to obtain a calibration result of the camera. 9. A camera calibration device, comprising a memory, a processor and a computer program stored on the memory, characterized in that: the processor executes the computer program to implement the method described in any one of claims 1 to 7 step. 10. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.