KR102238789B1 - Calibration method for triple camera-module - Google Patents

Abstract

The present invention relates to a calibration measurement method of a triple camera module that measures and provides error information of a triple camera module consisting of three lenses having three different wide angles, and a plurality of light sources having a set reference pattern are used as the triple camera module. Investigating; Capturing the plurality of light sources with the triple camera module to obtain the reference pattern images for each of the three lenses constituting the triple camera module, respectively; Calculating individual errors for each of the three lenses based on the reference pattern image; And calculating an angle difference between neighboring lenses among the three lenses based on individual errors for each of the three lenses.

Description

Calibration measurement method of triple camera module{CALIBRATION METHOD FOR TRIPLE CAMERA-MODULE}

The present invention relates to a calibration measurement method of a triple camera module, and more particularly, to a calibration measurement method of a triple camera module that calculates and provides error information between lenses for alignment of a triple camera module consisting of three lenses. .

In general, since the shape of an image acquired through a camera is determined according to the characteristics of the lens, it is necessary to determine the characteristics of the lens to configure the shape of an actual object appearing in the image.

In order to grasp the characteristics of the lens, a chart of a predefined pattern is used, and calibration is performed to estimate the characteristic parameter of the camera lens from the image of the specific chart.

In image analysis, camera calibration refers to a process of calculating a transformation matrix to obtain information on the actual size and position of an object detected in an image. The actual object exists in the 3D space, but since the object in the image exists in the 2D screen, distortion occurs naturally. That is, close objects on the screen appear larger and distant objects appear smaller.

Even if the distortion is corrected and displayed as small on the screen, if the actual size information of the small displayed object, that is, the width, height, and height in a three-dimensional space, can be obtained, much information can be obtained through image analysis. For this, a transformation matrix that can convert 2D coordinates of an image into actual 3D coordinates is required. That is, a transformation matrix that can convert the coordinates of (x, y) of the image into actual (X, Y, Z) is required.

Conventionally, in order to obtain such a transformation matrix, a reference object that already knows the width and length is photographed with a camera, and the coordinates of the reference object in the image are analyzed to perform camera calibration.

In the process, although the calibration program helped, there was a hassle of positioning the reference object at the place to be photographed and setting the coordinates of the reference object.

For example, looking at the invention of "Vehicle Stereo Camera Calibration Method and System" in Korean Patent Publication No. 10-1545633, it can be seen that a calibration marker detachable from the bonnet of the vehicle is used to calibrate the vehicle stereo camera. . As described above, in the prior art, there is a step that consumes time and cost, such as having to manually install a calibration marker each time calibration is performed.

Meanwhile, recently, a camera module is configured with two or more lenses and installed in a smartphone, and recently, a triple camera module composed of three lenses has been developed and installed in a terminal.

However, there is a limitation in that an error for a triple lens cannot be clearly calculated with a calibration system such as the prior art.

Accordingly, there is a need for a new technology capable of overcoming the problems of the prior art as described above.

Korean Registered Patent Publication No. 10-1545633

The present invention was created to improve the problems of the prior art as described above, and provides a calibration measurement method of a triple camera module capable of calculating and providing error information between lenses for alignment of a triple camera module consisting of three lenses. It is its purpose to provide.

Specifically, the present invention provides a method for measuring a calibration of a triple camera module capable of calculating an individual error of each lens based on a reference pattern image acquired through a plurality of light sources, and then calculating the angular difference between neighboring lenses. That is its purpose.

More specifically, it is an object of the present invention to provide a method for measuring a calibration of a triple camera module capable of calculating the angular difference between neighboring lenses by calculating the lateral distortion, the vertical distortion, and the rotational distortion of each lens. .

A method for measuring calibration of a triple camera module according to an embodiment of the present invention for achieving the above object is a triple camera module that measures and provides error information of a triple camera module composed of three lenses having different wide angles. A calibration measuring method of, comprising: irradiating a plurality of light sources having a set reference pattern with the triple camera module; Capturing the plurality of light sources with the triple camera module to obtain the reference pattern images for each of the three lenses constituting the triple camera module, respectively; Calculating individual errors for each of the three lenses based on the reference pattern image; And calculating an angle difference between neighboring lenses among the three lenses based on individual errors for each of the three lenses.

In addition, the plurality of light sources may include a center light source providing a reference for a rotation direction in the reference pattern image; A light source providing a reference in the left and right directions in the reference pattern image; And a left light source providing a reference in the vertical direction in the reference pattern image.

In addition, the calculating of the individual error may include calculating a horizontal distortion and a vertical distortion for each of the three lenses based on the center light source among the reference pattern images for each of the three lenses; And calculating a rotational direction shift for each of the three lenses based on the light source and the left light source among the reference pattern images for each of the three lenses.

In addition, the step of calculating the horizontal distortion and the vertical distortion may include: the horizontal and vertical lengths of the reference pattern image, the horizontal and vertical coordinates of the center light source, and the reference pattern image. Calculating a longitudinal separation distance and a lateral separation distance of the center light source from the center of the reference pattern image based on the pixel size; And calculating the horizontal and vertical distortion values for the lens based on the longitudinal separation distance and the lateral separation distance of the center light source, an optical axis distance value for the lens, and a radian value. It can be carried out including steps.

In addition, the calculating of the rotation direction distortion may include the light source and the left based on the vertical coordinates and the horizontal coordinates of the light source in the reference pattern image, and the vertical coordinates and the horizontal coordinates of the left light source. Calculating a longitudinal separation distance and a lateral separation distance between the light sources, respectively; And calculating the rotational direction distortion value for the lens based on the longitudinal separation distance, the lateral separation distance, and a radian value.

In addition, calculating the angular difference between the lenses may include calculating a difference in the horizontal distortion value between a pair of adjacent lenses among the three lenses; Calculating a difference in the vertical distortion value between a pair of adjacent lenses among the three lenses; And calculating a difference in the rotation direction distortion value between a pair of adjacent lenses among the three lenses.

According to the calibration measurement method of a triple camera module according to an embodiment of the present invention, after calculating individual errors of each lens based on a reference pattern image acquired through a plurality of light sources, the angle difference between neighboring lenses is calculated. It is possible to accurately provide error information for alignment of a triple camera module composed of three lenses.

Specifically, the present invention can provide a calibration measurement method of a triple camera module capable of calculating the angular difference between neighboring lenses by calculating the lateral twist, the vertical twist, and the rotational twist of each lens.

The effects that can be obtained in the disclosed embodiments are not limited to the above-mentioned effects, and other effects not mentioned are obvious to those of ordinary skill in the art to which the embodiments disclosed from the following description belong. It will be understandable.

1 is a flowchart illustrating a method of measuring calibration of a triple camera module according to an embodiment of the present invention.
2 is a flow chart specifically showing steps of calculating individual errors according to an embodiment of the present invention.
3 is a block diagram showing a triple camera module and a plurality of light sources according to the present invention.
4 is a block diagram showing a reference pattern image for explaining the steps of calculating the horizontal distortion and the vertical distortion according to the present invention.
5 is a block diagram showing a reference pattern image for explaining the step of calculating the rotational direction distortion according to the present invention.
6 is a flowchart specifically illustrating a step of calculating an angle difference between lenses according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In describing the present invention, detailed descriptions of related known general functions or configurations will be omitted.

Since the embodiments according to the concept of the present invention can apply various changes and have various forms, specific embodiments will be illustrated in the drawings and described in detail in the present specification or application. However, this is not intended to limit the embodiments according to the concept of the present invention to a specific form of disclosure, it should be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the present invention.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle. Other expressions describing the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to" should be interpreted as well.

The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of a set feature, number, step, action, component, part, or combination thereof, but one or more other features or numbers. It is to be understood that the possibility of addition or presence of, steps, actions, components, parts, or combinations thereof is not preliminarily excluded.

1 is a flow chart showing a calibration measurement method of a triple camera module according to an embodiment of the present invention, FIG. 2 is a flow chart specifically showing a step of calculating an individual error according to an embodiment of the present invention, and FIG. 3 is It is a configuration diagram showing a triple camera module and a plurality of light sources of the present invention. In addition, FIG. 4 is a configuration diagram showing a reference pattern image for explaining the steps of calculating the horizontal distortion and the vertical distortion according to the present invention, and FIG. 5 is a diagram illustrating the step of calculating the rotation direction distortion according to the present invention. It is a configuration diagram showing a reference pattern image for, and FIG. 6 is a flow chart specifically showing a step of calculating an angle difference between lenses according to the present invention.

In the method for measuring calibration of a triple camera module according to an embodiment of the present invention, as shown in FIG. 3, error information for alignment of the triple camera module 10 composed of three lenses 11 having different wide angles is obtained. It is a method of measuring and providing.

Specifically, the method of measuring the calibration of the triple camera module according to an embodiment of the present invention may be performed through an alignment measuring device not shown, and as shown in FIG. 1, the step of irradiating a light source (S100), a reference It may be performed including obtaining a pattern image (S200), calculating an individual error (S300), and calculating an angle difference between the lenses (S400).

The step of irradiating the light source (S100) is a step of irradiating a plurality of light sources 20 to the triple camera module 10 as shown in FIG. 3.

The step of irradiating such a light source (S100) may be performed by installing the triple camera module 10 in an alignment measuring apparatus, not shown, and irradiating a plurality of light sources 20 toward the triple camera module 10.

Here, the plurality of light sources 20 may be composed of three, and specifically, a center light source 21 providing a reference for a rotational direction in a reference pattern image 30 to be described later, and a reference in the left and right direction (lateral direction). It may be configured to include a light light source 22 to provide and a left light source 23 to provide a reference in the vertical direction (vertical direction).

The plurality of light sources 20 may irradiate light toward the triple camera module 10 while operating under the control of the alignment measuring device.

The step of obtaining the reference pattern image (S200) is a step of obtaining each of the reference pattern images 30 for each of the three lenses 11 constituting the triple camera module 10.

In the step of acquiring such a reference pattern image (S200), the plurality of light sources 20 are photographed with the triple camera module 10 through the control of the alignment measuring device while the light of the plurality of light sources 20 is irradiated. Reference pattern images 30 as shown in FIG. 30 may be obtained for each lens 11.

In this case, a plurality of light sources 20 may be photographed in a state in which the triple camera module 10 is set to be automatically exposed, and a reference pattern image 30 for each of the lenses 11 may be obtained.

That is, the reference pattern image 30 may display the center light source 21, the right light source 22, and the left light source 23, respectively, while having a set width and height.

The step of calculating the individual error (S300) is a step of calculating the individual error of each of the lenses 11 based on the reference pattern image 30 of each of the obtained lenses 11.

Specifically, the step of calculating the individual error (S300) is a step of calculating the distortion in the left and right directions and the distortion in the vertical direction for each of the lenses 11 as shown in FIG. 2 (S310) and calculating the rotation direction distortion It may be performed including (S320).

The step of calculating the horizontal distortion and the vertical distortion (S310) is performed in the horizontal direction for each of the lenses 11 based on the center light source 21 displayed in the reference pattern image 30 for each of the three lenses 11. (Transverse direction) distortion and vertical direction (longitudinal distortion) can be calculated respectively.

The step (S310) of calculating such a left-right twist and a vertical twist is a step (S311) of calculating the vertical separation distance (dx1) and the horizontal separation distance (dy1) of the center light source (S311), and the horizontal distortion value and the vertical separation. It may be performed including the step (S312) of each calculating the direction distortion value.

Ultimately, the step (S311) of calculating the longitudinal separation distance (dx1) and the lateral separation distance (dy1) of the center light source, respectively, is a center for the center of the reference pattern image 20 (Width/2, Height/2). The longitudinal separation distance dx1 and the lateral separation distance dy1 of the light source 21 may be calculated.

Specifically, the step (S311) of calculating the longitudinal separation distance dx1 and the lateral separation distance dy1 of the center light source, respectively, is the transverse length of the reference pattern image 30 obtained as shown in FIG. 4 ( width) and vertical length (height), the horizontal coordinate (cx) and the vertical coordinate (cy) of the center light source 21 displayed in the reference pattern image 30, the pixel size for the reference pattern image, and the reference It may be performed through the center of the pattern image 20 (Width/2, Height/2).

Here, the step (S311) of calculating each of the longitudinal separation distance dx1 and the lateral separation distance dy1 of the center light source may be performed through the following [Equation 1].

[Equation 1]

dx1 = (Width/2-cx) * PixelSize

dy1 = (Height/2-cy) * PixelSize

(Where, Width is the horizontal length of the reference pattern image, Height is the vertical length of the reference pattern image, cx is the horizontal coordinate of the center light source, cy is the vertical coordinate of the center light source, and PixelSize is the pixel size of the reference pattern image. )

Accordingly, the longitudinal separation distance (dx1) and the lateral separation distance (dy1) of the center light source 21 with respect to the center (Width/2, Height/2) of the reference pattern image 20 can be obtained from [Equation 1]. It can be calculated and stored through.

The step of calculating the horizontal distortion value and the vertical distortion value (S312) is a step of calculating the horizontal distortion value (Theta_X) and the vertical direction distortion value (Theta_Y) for each lens 11, respectively, as described above. It may be calculated based on the longitudinal separation distance dx1 and the lateral separation distance dy1 calculated in step S311, the optical axis distance value EFL for the lens, and the radian value.

Here, the step (S312) of respectively calculating the horizontal distortion value Theta_X and the vertical distortion value Theta_Y may be performed through [Equation 2] below.

[Equation 2]

Theta_X = arctan(dx1 / EFL) * (180/3.14)

Theta_Y = arctan(dy1 / EFL) * (180/3.14)

(Here, EFL is the optical axis distance value for the lens)

That is, the horizontal distortion value (Theta_X) and the vertical distortion value (Theta_Y) with respect to the lens 11 are the calculated longitudinal separation distances (dx1) and transverse separation distances (dy1) calculated by [Equation 1]. , It may be calculated as an inverse tangent function based on the constant optical axis distance value (EFL) of the lens 11 and the Rian value (180/3.14).

In the step of calculating the rotational distortion (S320), rotation for each of the lenses 11 based on the light source 22 and the left light source 23 among the reference pattern images 30 for each of the three lenses 11 You can calculate the skew.

The step of calculating such rotational distortion (S320) is a step of calculating the longitudinal separation distance (dx2) and the lateral separation distance (dy2) between the right light source 22 and the left light source 23, respectively (S321) and rotation. It may be performed including the step (S322) of calculating the direction distortion value.

Specifically, the step (S321) of calculating the longitudinal separation distance (dx2) and the lateral separation distance (dy2) between the right light source 22 and the left light source 23, respectively, is obtained as shown in FIG. In the pattern image 30, the horizontal coordinates (rx) and vertical coordinates (ry) of the light source 22, and the horizontal coordinates (lx) and the vertical coordinates (ly) of the left light source 23 I can.

Here, the step (S321) of calculating each of the longitudinal separation distance (dx2) and the lateral separation distance (dy2) between the light light source 22 and the left light source 23 may be performed through the following [Equation 3]. have.

[Equation 3]

dx2 = (rx-lx) * PixelSize

dy2 = (ry-ly) * PixelSize

(Where rx is the horizontal coordinate of the light source, ry is the vertical coordinate of the light source, lx is the horizontal coordinate of the left light source, ly is the vertical coordinate of the left light source, and PixelSize is the pixel size of the reference pattern image)

Accordingly, the longitudinal separation distance dx2 and the lateral separation distance dy2 between the right light source 22 and the left light source 23 may be calculated and stored through [Equation 3].

The step of calculating the rotation direction distortion value (S322) is a step of calculating the rotation direction distortion value (Theta_R) for each lens 11, and the light source 22 and the left light source 23 calculated in the above-described step S321. ) Can be calculated based on the longitudinal separation distance (dx2) and the lateral separation distance (dy2) between, and the radian value.

Here, the step (S322) of calculating the rotational direction distortion value Theta_R may be performed through the following [Equation 4].

[Equation 4]

Theta_R = arctan(dx2 / dy2) * (180/3.14)

(Here, EFL is the optical axis distance value for the lens)

That is, the rotational distortion value (Theta_R) for the lens 11 is the longitudinal separation distance (dx2) and the lateral separation between the light source 22 and the left light source 23 calculated by [Equation 3]. The distance dy2 and an inverse tangent function based on a constant Rian value (180/3.14) may be respectively calculated.

Accordingly, the vertical distortion value (Theta_X), the vertical distortion value (Theta_Y), and the rotation direction distortion value (Theta_R) for each of the three lenses 11 are calculated and stored. Can be calculated.

The step of calculating the angular difference between the lenses (S400) is a pair of lenses 11 adjacent to each other among the three lenses 11 based on the calculated individual errors (Theta_X, Theta_YnTheta_R) for each of the lenses 11 This is the step of calculating the angular difference between them.

That is, in the step of calculating the angular difference between the lenses (S400), angular differences for three adjacent pairs of the three lenses 11 may be calculated, respectively.

Specifically, the step of calculating the angular difference between the lenses (S400) is a difference between the step (S410) of calculating the difference between the left-right distortion value (Theta_X) and the vertical direction distortion value (Theta_Y) as shown in FIG. It may be performed including the step of calculating (S420) and the step (S430) of calculating the difference between the rotational direction twist value (Theta_R).

The step of calculating the difference between the left and right distortion values Theta_X (S410) is by calculating the difference between the horizontal distortion values Theta_X between the pair of lenses 11 adjacent to each other among the three lenses 11 While storing, the difference between the left and right twist values (Theta_X) for the three pairs of the three lenses 11 may be calculated and stored.

The step of calculating the difference between the vertical distortion value Theta_Y (S420) is by calculating the difference between the vertical distortion value Theta_Y between the pair of lenses 11 adjacent to each other among the three lenses 11 While storing, the difference between the vertical distortion values Theta_Y for the three pairs of the three lenses 11 may be calculated and stored.

In the step (S430) of calculating the difference between the rotation direction distortion value Theta_R, the difference between the rotation direction distortion value Theta_R between a pair of adjacent lenses 11 among the three lenses 11 is calculated. While storing, the difference between the rotation direction distortion values Theta_R for the three pairs of the three lenses 11 may be calculated and stored.

The difference between the horizontal distortion value (Theta_X), the vertical distortion value (Theta_Y), and the rotation direction distortion value (Theta_R) between the lenses 11 calculated through the above process is not shown. It can be used to align the lenses 11 by being provided on.

As described above, in the calibration measurement method of the triple camera module according to an embodiment of the present invention, after calculating individual errors of each lens based on a reference pattern image acquired through a plurality of light sources, the angle between neighboring lenses By calculating the difference, error information for alignment of the triple camera module composed of three lenses can be accurately provided.

The above-described embodiments are for illustrative purposes only, and those of ordinary skill in the art to which the above-described embodiments belong can easily transform into other specific forms without changing the technical spirit or essential features of the above-described embodiments. You can understand. Therefore, it should be understood that the above-described embodiments are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope to be protected through the present specification is indicated by the claims to be described later rather than the detailed description, and should be interpreted as including all changes or modified forms derived from the meaning and scope of the claims and the concept of equivalents thereof. .

10: triple lens module
11: lens
20: multiple light sources
21: Center light source
22: light source
23: Left light source
30: reference pattern image

Claims (7)

As a calibration measurement method of a triple camera module that measures and provides error information of a triple camera module consisting of three lenses with different wide angles,
Irradiating a plurality of light sources having a set reference pattern with the triple camera module;
Capturing the plurality of light sources with the triple camera module to obtain the reference pattern images for each of the three lenses constituting the triple camera module, respectively;
Calculating individual errors for each of the three lenses based on the reference pattern image; And
Comprising the step of calculating an angle difference between neighboring lenses among the three lenses based on individual errors for each of the three lenses,
The plurality of light sources,
A center light source providing a reference for a rotation direction in the reference pattern image;
A light source providing a reference in the left and right directions in the reference pattern image; And
Including a left light source providing a reference in the vertical direction in the reference pattern image,
The step of calculating the individual error,
Calculating a horizontal distortion and a vertical distortion for each of the three lenses based on the center light source among the reference pattern images for each of the three lenses; And
And calculating a rotational direction shift for each of the three lenses based on the light source and the left light source among the reference pattern images for each of the three lenses.
delete delete The method of claim 1,
The step of calculating the horizontal distortion and the vertical distortion,
The length of the center light source from the center of the reference pattern image based on the horizontal and vertical length of the reference pattern image, the horizontal and vertical coordinates of the center light source, and the pixel size of the reference pattern image. Calculating a directional separation distance and a lateral separation distance, respectively; And
Calculating the horizontal and vertical distortion values for the lens based on the longitudinal separation distance and the lateral separation distance of the center light source, an optical axis distance value for the lens, and a radian value. Calibration measurement method of the triple camera module comprising a.
The method of claim 4,
The step of calculating the longitudinal separation distance and the lateral separation distance of the center light source, respectively, may include a longitudinal separation distance (dx1) and a lateral separation distance of the center light source from the center of the reference pattern image by the following [Equation 1]. Calculate distance (dy1) respectively,
The step of calculating the horizontal distortion value and the vertical distortion value respectively comprises calculating the horizontal distortion value (Theta_X) and the vertical direction distortion value (Theta_Y) according to the following [Equation 2]. How to measure the calibration of the triple camera module.
[Equation 1]
dx1 = (Width/2-cx) * PixelSize
dy1 = (Height/2-cy) * PixelSize
(Where, Width is the horizontal length of the reference pattern image, Height is the vertical length of the reference pattern image, cx is the horizontal coordinate of the center light source, cy is the vertical coordinate of the center light source, and PixelSize is the pixel size of the reference pattern image. )
[Equation 2]
Theta_X = arctan(dx1 / EFL) * (180/3.14)
Theta_Y = arctan(dy1 / EFL) * (180/3.14)
(Here, EFL is the optical axis distance value for the lens) The method of claim 1,
The step of calculating the rotational direction distortion,
In the reference pattern image, based on the vertical coordinates and the horizontal coordinates of the light light source, and the vertical coordinates and the horizontal coordinates of the left light source, a vertical separation distance and a horizontal separation distance between the light light source and the left light source Calculating each; And
And calculating the rotation direction distortion value for the lens based on the longitudinal separation distance, the lateral separation distance, and a radian value.
The method of claim 6,
The step of calculating the longitudinal separation distance and the lateral separation distance, respectively, is a longitudinal separation distance (dx2) and a lateral separation distance (dy2) between the light source and the left light source by the following [Equation 3] Calculate,
In the calculating of the rotation direction distortion value, the calibration measurement method of a triple camera module, characterized in that the rotation direction distortion value (Theta_R) is calculated by the following [Equation 4].
[Equation 3]
dx2 = (rx-lx) * PixelSize
dy2 = (ry-ly) * PixelSize
(Where rx is the horizontal coordinate of the light source, ry is the vertical coordinate of the light source, lx is the horizontal coordinate of the left light source, ly is the vertical coordinate of the left light source, and PixelSize is the pixel size of the reference pattern image)
[Equation 4]
Theta_R = arctan(dx2 / dy2) * (180/3.14)

Images