[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Next Article in Journal
A Game-Based Computing Resource Allocation Scheme of Edge Server in Vehicular Edge Computing Networks Considering Diverse Task Offloading Modes
Next Article in Special Issue
Combined Displacement and Angle Sensor with Ultra-High Compactness Based on Self-Imaging Effect of Optical Microgratings
Previous Article in Journal
Design and Development of Large-Band Dual-MSFA Sensor Camera for Precision Agriculture
Previous Article in Special Issue
Jitter-Caused Clutter and Drift-Caused Clutter of Staring Infrared Sensor in Geostationary Orbit
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 24
Issue 1
10.3390/s24010066
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Vision Measurement Method Based on Plate Glass Window Refraction Model in Tunnel Construction

by
Zhen Wu
1,
Junzhou Huo
1,2,*,
Haidong Zhang
1,
Fan Yang
1,
Shangqi Chen
1 and
Zhihao Feng
1
1
School of Mechanical Engineering, Dalian University of Technology, Dalian 116024, China
2
State Key Laboratory of High-Performance Precision Manufacturing, Dalian 116024, China
*
Author to whom correspondence should be addressed.
Sensors 2024, 24(1), 66; https://doi.org/10.3390/s24010066
Submission received: 5 November 2023 / Revised: 15 December 2023 / Accepted: 18 December 2023 / Published: 22 December 2023
(This article belongs to the Special Issue Imaging and Sensing in Optics and Photonics)
Browse Figures
Figure 1
<p>The TBM diagram of the changing area.</p> "> Figure 2
<p>The basic content of this article.</p> "> Figure 3
<p>An Imaging model when the camera’s optical axis is not parallel to the normal vector of the refraction plane. (<b>a</b>) Plane refraction imaging and (<b>b</b>) image points on the imaging surface.</p> "> Figure 4
<p>Non-parallel plane refractive camera rotation model.</p> "> Figure 5
<p>Non-parallel plane refractive camera imaging model.</p> "> Figure 6
<p>Independent refractive surface (<b>a</b>) imaging model and (<b>b</b>) epipolar constraint model.</p> "> Figure 7
<p>Triangulation measurement model under an independent refraction plane.</p> "> Figure 8
<p>Overall flow chart of binocular measurement under plate refraction.</p> "> Figure 9
<p>Experimental platform for vision program of the tool-change robot.</p> "> Figure 10
<p>Image deviation distribution plot. (<b>a</b>) Glass attitude 1 and (<b>b</b>) glass attitude 2.</p> "> Figure 11
<p>Determination of the distortion center. (<b>a</b>) Glass attitude 1 and (<b>b</b>) glass attitude 2.</p> "> Figure 12
<p>Image point comparison before and after camera rotation with glass. (<b>a</b>) Glass attitude 1 and (<b>b</b>) glass attitude 2.</p> "> Figure 13
<p>Pinhole camera model versus refractive imaging model positioning error.</p> "> Figure 14
<p>The measured segment position.</p> "> Figure 15
<p>Determination of the distortion center. (<b>a</b>) Left camera and (<b>b</b>) right camera.</p> "> Figure 16
<p>Epipolar correction diagram with glass.</p> "> Figure 17
<p>Matching error with and without refraction at different glass positions.</p> "> Figure 18
<p>Comparison before and after calibration. (<b>a</b>) Depth of field and (<b>b</b>) depth of field error and positioning error.</p> "> Figure 19
<p>Pollution type diagram. (<b>a</b>) Glass is polluted by sewage. (<b>b</b>) Glass is polluted by sludge.</p> ">
Review Reports Versions Notes

Abstract

:
Due to the harsh environment of high humidity and dust in tunnel construction, the vision measurement system needs to be equipped with an explosion-proof glass protective cover. The refractive effect of the plate glass window invalidates the pinhole model. This paper proposes a comprehensive solution for addressing the issue of plane refraction. First, the imaging model for non-parallel plane refraction is established based on dynamic virtual focal length and the Rodriguez formula. Further, due to the failure of the epipolar constraint principle in binocular vision systems caused by plane refraction, this paper proposes the epipolar constraint model for independent refractive plane imaging. Finally, an independent refraction plane triangulation model is proposed to address the issue of triangulation failure caused by plane refraction. The RMSE of the depth of field errors in the independent refraction plane triangulation model is 2.9902 mm before correction and 0.3187 mm after correction. The RMSE of the positioning errors before and after correction are 3.5661 mm and 0.3465 mm, respectively.

1. Introduction

A Tunnel Boring Machine (TBM) is an extensive piece of equipment utilized in tunnel construction. It combines various functions including excavation, removal of debris, and lining. TBMs are widely used in many fields such as railroads, highways, water diversion projects, and municipal pipelines. In the TBM digging process, the cutter is subjected to complex and variable loads. The cutter consumes a lot during the construction process and requires frequent replacement [1,2]. The tool change area [3] is shown in Figure 1. The cutter change area is very narrow and not conducive to manual work. At present, TBM tool-changing work is dangerous and inefficient [4]. This has led to an urgent demand for robotic tool-changing technology in the field of tunnel construction. Currently, manual operations are primarily relied upon for tool detection and replacement due to limitations in technological development. During the construction process of TBMs, there are considerable safety hazards related to manual operations in harsh construction environments. These environments include deep burial, high water pressure, and high levels of dust, which can potentially result in serious safety accidents such as casualties. Thus, achieving intelligent tool-changing in harsh and complex tunnel construction environments requires precise positioning and accurate measurement of the tool system.
Machine vision measurement technology has advantages over other measurement technologies such as non-contact measurement, remote manipulation, information-rich, fast response, and accuracy [5,6,7]. The environment of the TBM tool change bin is often characterized by high pressure [8], high humidity, and dust. To ensure safety, the camera inside the tool changing bin needs to be equipped with an explosion-proof glass protective cover [9,10]. However, the traditional pinhole camera model cannot eliminate the refractive distortion caused by glass. The existing distortion models with a single factor (only considering the lens) cannot describe the imaging patterns of cameras under planar refraction [11].
This paper presents a comprehensive solution for binocular measurement under the refraction of plate glass. The specific contributions are as follows:
(1)
The non-parallel plane refractive camera imaging model is established based on dynamic virtual focal length and the Rodriguez formula.
(2)
The main problems are the epipolar constraint failure and triangulation failure under refraction images. This paper proposes an epipolar constraint model and triangulation method based on independent refraction planes.
The remainder of this paper is structured as follows. Section 2 introduces the related work. Section 3 describes, in detail, the model for image distortion correction under non-parallel plane refraction, the model for imaging epipolar constraint on independent refractive surfaces, and the triangulation method for independent refraction planes. Section 4 validates the effectiveness of the proposed method by conducting a series of physical experiments. Section 5 discusses the relevant issues. Section 6 presents the conclusion.

2. Related Work

The refraction effect of glass severely affects the accuracy of visual measurements. Scholars have conducted in-depth research on the characterization and elimination of refractive distortion. In the calibration of visual measurement system parameters under planar refraction, Treibitz [12] studied the trajectory of camera viewpoints under planar refraction environments. In addition, Treibitz proposed an imaging model for a camera based on dynamic virtual focal length. The measurement accuracy of this model is significantly greater compared to single-viewpoint models. However, this model requires prior knowledge of the depth information of the target object. The camera’s optical axis must also be parallel to the normal vector of the refractive surface. Agrawal [13] established an imaging model for multiplane refraction. The multi-plane refraction imaging model has been proven to have strong similarities to the single-plane refraction imaging model.
Chen [14] introduced a calibration method for determining the normal vector of the refractive surface as well as the thickness of the glass. The experiment demonstrates that this method exhibits high accuracy and robustness. Shimizu [15,16] proposed a camera depth estimation method that utilizes the parallel plane refraction effect. However, this method requires a complex calibration setup. Gong [17] proposed a 3D reconstruction model based on plane refraction correction. This method applies to the scene of any number and direction of plate glass. He also proposed a method for flexibly determining the normal vector of the refractive surface without any auxiliary devices. Ke [18] examined how the thickness of a glass plate affects the resulting image. He used a target extremely close to the plate glass to obtain the normal vector of the refractive surface and the distance between the plate glass and the camera’s optical center. Huang S [19] proposed a multi-camera calibration method based on planar refraction. Multiple cameras can capture calibration images from different angles. This calibration method has high accuracy. The basic geometric relationships based on plane refraction can be used to eliminate refraction errors. Shortis et al. [20] employ corrective lenses or dome glass to eliminate refractive distortion, guaranteeing that the light rays from every point are perpendicular to the refractive surface. This situation can still use a single viewpoint model. However, this method requires extremely high accuracy. In practical applications, manufacturing errors and installation deviations of lenses often lead to other refractive errors.
Additionally, scholars have conducted research on stereo matching in binocular vision under plane refraction. Yamashita [21] considered the refraction effect and proposed a model for underwater 3D reconstruction. However, he did not consider the issue of polar constraint failure under refractive effects. Huang et al. [22] proposed a multi-plane refractive imaging model. He proposed the theory of dynamic epipolar constraint, which solved the problem that the epipolar of stereo matching under plane refraction is a curve. This method verifies the accuracy of matching points. However, the camera’s optical axis remains parallel to the normal vector of the refractive surface. Gedge [23] studied the effect of planar refraction on underwater imaging and stereo matching. He proposed a 3D reconstruction model when the camera’s optical axis is not parallel to the normal vector of the refractive surface, and the curve equations for the epipolar constraint in stereo matching are calculated.
In summary, a situation of approximate correction in the research of visual measurement methods under plane refraction remains. Many scholars have built plane refraction imaging models that are too complex, with too many introduced external parameters. The model is not concise and requires large computational efforts.
This paper investigates the problem of error correction for vision measurement systems considering plane refraction and presents a complete approach to visual measurements. This method includes image distortion correction, refraction plane normal vector solving, polar constraint modeling of refraction images, triangulation modeling of refraction imaging, and refraction error measurement.

3. Methodology

The traditional pinhole camera model based on perspective imaging leads to vision measurement errors due to refraction distortion. A new refractive imaging model needs to be established to eliminate refractive errors. Ideally, the camera’s optical axis is parallel to the normal vector of the refractive plane. Due to installation and manufacturing errors, the camera’s optical axis is not guaranteed to be parallel to the normal vector of the refraction plane. An image distortion correction model is established based on the concept of dynamic virtual focal length in a parallel plane. It complements the non-parallel plane camera rotation model to achieve correction. The main flow of this paper is shown in Figure 2.

3.1. Two-Dimensional Measurement with Monocular Vision under Non-Parallel Plane Refraction

3.1.1. Finding the Normal Vector of a Refraction Plane

In Figure 3, the camera’s optical axis is not parallel to the normal vector n of the refractive plane. Based on the expansion offset effect of plane refraction, the normal vector of the refraction plane of the plate glass is determined.
The camera coordinate system is established with O as the origin. The Mg1(ug1, vg1) and Mg2(ug2, vg2) points are the refracted imaging points. M1(u1, v1) and M2(u2, v2) are the image points without refraction. Mc(uc, vc) is the distortion center point. One can maintain the calibration board posture and take two separate images, one directly imaged and the other imaged through glass. These two images can then be used to get the refraction plane normal vector. As depicted in Figure 3a, the vector McO is parallel to the normal vector n of the refraction plane in the camera coordinate system. As depicted in Figure 3b, the intersection of two straight lines in the ideal state is the distortion center point. The intersection point between the normal vector of the glass refractive plane passing through the optical center and the image plane is the distortion center. Therefore, different glass attitudes can cause significant differences in the location of distortion centers. In practical implementation, it is necessary to calculate multiple lines to ensure the accuracy of the distortion center. The point closest to all the lines is taken as the distortion center.
The equation of the line connecting the refracted image point and the non-refracted image point is:
x ( v g i − v i ) + y ( u i − u g i ) + v i u g i − u i v g i = 0
The distortion center Mc(uc, vc) is the point closest to all straight lines, which can be solved by linear least square method [23]. The objective function is established as follows:
arg min ( u c , v c ) ∑ i = 1 K u c ( v g i − v i ) + v c ( u i − u g i ) + v i u g i − u i v g i ( v g i − v i ) , ( u i − u g i ) 2 2
The normal vector of the refraction plane of the plate glass is
n = [ u c − u 0 f d x , v c − v 0 f d y , 1 ]

3.1.2. Modeling of Camera Rotation under Non-Parallel Plane Refraction

The image distortion correction model under parallel plane refraction is established based on the dynamic virtual focal length. After determining the refraction plane normal vector n, the position of the inclined plate glass in the camera coordinate system can be accurately described. It can establish a camera rotation model under non-parallel plane refraction. In Figure 4, the virtual camera coordinate system (blue dotted line in the figure) is established with point O as the origin. The tilt of plate glass can be viewed as a rotation centered on the rotation axis past the origin O.
The unit vector v of the rotation axis can be found from the camera optical axis direction vector w and the refraction plane normal vector n as follows:
v = w × n w × n
The angle of rotation is as follows:
cos λ = w ⋅ n w ⋅ n
The virtual camera coordinate system can be obtained by rotating the real camera coordinate system around the rotation axis v using λ angle. First, the Qz(xz, yz, zz) of the camera coordinate system is transformed into the virtual camera coordinate system. It follows from the Rodriguez equation:
q z ′ = sin λ ( v × q z ) + ( 1 − cos λ ) ( v ⋅ q z ) v + q z cos λ
where qz = [xz, yz, zz]T, qz′ = [qz1, qz2, qz3]. The qz′ is the coordinate of Qz in the virtual camera coordinate system. In the virtual camera coordinate system, the intersection of the line OQz with the virtual image plane is the image point Q2(x2, y2, z2).
x 2 q z 1 = y 2 q z 2 = z 2 q z 3 z 2 = − f
The following is from Equation (7):
x 2 = − f ⋅ q z 1 q z 3 y 2 = − f ⋅ q z 2 q z 3
In conclusion, the coordinates of the image point in the virtual camera coordinate system can be obtained. The correction for non-parallelism between the camera optical axis and the normal vector of the refracting surface can be realized.

3.1.3. Modeling of Camera Imaging under Non-Parallel Plane Refraction

In Figure 5, the intersection point between the camera optical axis and the measurement plane in the camera coordinate system is Q0. Q1 is any point on the measurement plane. The image point of Q1 refracted by the plane on the virtual imaging plane is Q2. The main point of the virtual camera is Q4. The incident angle is θa1. The distance between line OQ2 and line L is denoted as dw1. In the camera coordinate system, the coordinates of Q0 are (0, 0, h). In the virtual camera coordinate system, the coordinates of Q0, Q1, Q2, Q3, and Q4 are Q0(x0, y0, z0), Q1(x1, y1, z1), Q2(x2, y2, z2), Q3(x3, y3, z3), and Q4(x4, y4, z4). The coordinates of Q0 and Q2 in the virtual camera coordinate system can be determined based on the camera rotation model. The coordinates of Q4 in the virtual camera coordinate system are (0, 0, −ƒ). In the camera coordinate system, the normal vector of the refractive surface is n = [n1, n2, n3]. The normal vector of the measurement plane is w = [0, 0, 1]. The w can be corrected by the camera rotation model to obtain the normal vector of the measurement plane in the virtual camera coordinate system as w1 = [−n1, −n2, n3].
In the virtual camera coordinate system, the normal vector w1 of the measurement plane and a certain point Q0(x0, y0, z0) in the measurement plane are known. From any point Q(x, y, z) in the measurement plane:
Q Q 0 ⇀ ⋅ w 1 = 0
The point Q coordinate satisfies the following:
a ( x − x 0 ) + b ( y − y 0 ) + c ( z − z 0 ) = 0
After this, the equation of the measurement plane can be obtained as follows:
a x + b y + c z + d = 0
where a = −n1, b = −n2, c = n3, d = −(ax0 + by0 + cz0).
In the virtual camera coordinate system, the distances Q2Q4 and Q3Q4 on the virtual imaging surface are r2 and r3, respectively.
This can be obtained from the geometric relationship in Figure 5:
tan θ a 1 = r 2 f d w 1 = d sin θ a 1 ( 1 − 1 − sin 2 θ a 1 n p 2 − sin 2 θ a 1 ) r 3 − r 2 = d w 1 cos θ a 1
Since the line Q2Q4 is collinear with the line Q3Q4, it follows:
x 2 x 3 = y 2 y 3
By combining Equations (12) and (13), x3 and y3 can be obtained. The coordinates of Q3 can be obtained.
In the virtual camera coordinate system, the line OQ2 is parallel to the line L. In Figure 5, the equation for line L is as follows:
x − x 3 x 2 = y − y 3 y 2 = z − z 3 z 2
The intersection point between the line L and the measurement plane is Q1. The coordinates of Q1 in the virtual camera coordinate system can be obtained by combining Equations (11) and (14). Its converted coordinate in the camera coordinate system is as follows:
q 1 = − sin λ ( v × q 1 ′ ) + ( 1 − cos λ ) ( v ⋅ q 1 ′ ) v + q 1 ′ cos λ
Here, q1′ = [x1, y1, z1]T, q1 = [q11, q12, q13], (q11, q12, q13) are the coordinates of Q1 in the camera coordinate system.

3.2. Epipolar Constraint Modeling of Binocular Vision System under Nonparallel Plane Refraction

Based on the geometric relationship of plane refraction, the epipolar constraint model of the binocular vision system under non-parallel plane refraction is derived. The virtual camera coordinate system is introduced, and the conversion relationship between the camera coordinate system and the virtual camera coordinate system is shown in Figure 5. The coordinate system of the left camera is denoted as OlXlYlZl. The coordinate system of the left virtual camera is denoted as OlXclYclZcl. The coordinate system for the right camera is denoted as OrXrYrZr. The coordinate system for the right virtual camera is denoted as OrXcrYcrZcr. The optical centers of the two cameras are Ol and Or. The base distance between the two optical centers is a. The focal length of the cameras is ƒ. The left and right virtual imaging planes are maintained parallel to the plate glass.
In Figure 6b, the two camera refraction planes are independent as shown. Based on the virtual camera coordinate system, the epipolar constraint model of independent refraction plane imaging is established. The distortion centers of the left and right virtual imaging planes are denoted as Fcl and Fcr, respectively. The thickness of plate glass is d. Ol is the origin of the left virtual camera coordinate system OlXclYclZcl. The left virtual camera coordinate system serves as the world coordinate system. F1(x1, y1, z1) is any point on the left virtual imaging plane. The line OlF1 intersects the front of the glass at point F2, and after being refracted by the left plate glass, intersects the back of the glass at point F3. After this, take any point Fh (xh, yh, zh) on the refraction ray and set zh = h. The inverse extension of the line F3Fh intersects the left virtual imaging plane at point F4. Set the refraction planes in the left and right cameras as Π1, Π2. As a result, the intersection of the refraction plane Π1 with the left virtual imaging plane is the line F1F4. The distortion center Fcl (xcl, ycl, zcl) of the left virtual imaging plane is on line F1F4. The coordinates of F4 is (x4,y4,z4). The incident angle is θa2. The lengths of lines FclF1 and FclF4 are R1 and R2, respectively. The distance between the incident and refraction rays is dw2.
From the geometric relationship in Figure 6:
R 1 2 = ( x 1 − x c l ) 2 + ( y 1 − y c l ) 2 R 2 2 = ( x 4 − x c l ) 2 + ( y 4 − y c l ) 2 tan θ a 2 = R 1 f d w 2 = d sin θ a 2 ( 1 − 1 − sin 2 θ a 2 n p 2 − sin 2 θ a 2 ) R 1 − R 2 = d w 2 cos θ a 2 x 1 x 4 = y 1 y 4 = z 1 z 4 z 1 = z 4 = z c l = f
where the coordinate of Fcl is (0, 0, ƒ). After calculating the coordinates of point F4, then it is obtained from the point F4 and Fh are co-linear:
x 4 x h = y 4 y h = z 4 z h z 4 = f z h = h
From Equations (16) and (17), the coordinates of points F4 and Fh can be obtained in the same way. In the right virtual camera coordinate system, the coordinate of the right camera optical center on the left virtual camera coordinate system is Or(a, 0, 0). After this, it can be obtained as F9(x9, y9, z9):
x 9 − a x h − a = y 9 y h = z 9 z h z 9 = f
The extension of the line FhF5 intersects the right virtual imaging surface at point F8. The points F7, F8, F9, and the center of distortion Fcr are collinear, and the line of this collinearity is Lr1. Therefore, it can be obtained that:
x 8 − x 9 x c r − x 9 = y 8 − y 9 y c r − y 9 = z 8 − z 9 z c r − z 9 y 8 = y 4 z 8 = z 4 = f
The solution for the coordinates of the distortion center Fcr in Equation (19) can be referred to Section 3.1.1. After calculating the coordinates of F8(x8, y8, z8), the coordinates of F7(x7, y7, z7) can be obtained in the same way. Any point on the left virtual imaging plane can obtain the corresponding epipolar line on the right virtual imaging plane.

3.3. Triangulation Modeling of Binocular Vision System under Non-Parallel Plane Refraction

The traditional triangulation method based on a pinhole camera model is ineffective in binocular vision measurement due to the plane refraction effect. This section analyzes the binocular vision measurement system under independent refractive planes. The image points of any space point P on the left and right imaging planes are Pl and Pr, respectively. The refraction rays OlPl and OrPr are not necessarily in the same plane. This leads to the failure of the conventional triangulation method based on the pinhole camera model. The camera rotation model and the camera imaging model under non-parallel plane refraction are proposed in Section 3.1.2. The model corrects the refraction error under non-parallel plane refraction. On this basis, this paper establishes a binocular vision system triangulation model under independent refractive planes.
In Figure 7, when the two refractive surfaces are independent, the camera coordinate system should be first corrected with rotation. A triangulation model of a binocular vision system with an independent refraction plane is established based on the virtual camera coordinate system. The distortion centers of the left and right virtual imaging surfaces are Wcl and Wcr, respectively. The image points of any space point W on the left and right virtual imaging surfaces are points W1 and W7, respectively. The reverse extension of line W3W intersects with the left virtual imaging surface at point W4. The reverse extension of line W5W intersects with the right virtual imaging surface at point W8. Triangulation modeling of binocular vision systems with independent refraction planes is based on a virtual camera coordinate system. From the previous analysis, points W, W4, and W8 are coplanar.
Triangular measurement of a binocular vision system under independent refractive surfaces can be achieved by using ray coplanarity as a constraint. The coordinates of points W4 and W8 can be solved from Equation (16).

3.4. Three-Dimensional Measurement Solution Process under Plane Refraction

The main flow of this paper is shown in Figure 8. The refraction plane normal vector n is solved by capturing before and after images of the refraction at the same attitude of the calibration plate. The camera rotation model is established based on the Rodriguez formula. The camera coordinate system is rotated and corrected to obtain the virtual camera coordinate system. The coordinates of Q1 in the virtual space coordinate system can be obtained by combining the equations of the measurement plane and the line L in the virtual camera coordinate system. The coordinates in the camera coordinate system are obtained from Equation (15). Both the left and right imaging planes undergo rotation correction, and binocular measurement is accomplished by employing the epipolar constraint model and the triangulation model under refraction effects.
The corner coordinates of the checkerboard image under the refraction effect are known, and the corresponding coordinate points in the virtual coordinate system can be obtained through simultaneous Equations (12) and (13). In Figure 7, the W1 and W7 coordinate points under the virtual coordinate system can be obtained. Further, the point coordinates of W4 and W8 in the virtual coordinate system can be solved by Equation (16) in Section 3.2. Finally, binocular measurements are realized using the refraction plane imaging epipolar constraint modeling and the triangulation model.

4. Experimentation and Analysis

A series of physical experiments were conducted to validate the effectiveness of the proposed method. Section 4.1 presented the experimental setup. Section 4.2 conducted experimental verification of camera imaging models under non-parallel plane refraction. Section 4.3 studied the 3D measurement experiment of binocular vision under plane refraction. The camera calibration was performed using the most common Zhang’s calibration method [24,25].

4.1. Experimental Setups

To validate the method proposed in this paper, a set of experimental platforms for the vision scheme of the tool change robot was built [26]. This is shown in Figure 9. It consists of a binocular camera, plate glass, target, motion module, and robot. Details are as follows:
(1)
The camera model is Cognex CIC-1300, which boasts a resolution of 1280 pixels × 1024 pixels. The focal length of the camera lens is 12.5 mm, and the field of view angle is 25°. The measurement volume is 0.08 m3. The camera baseline distance is 205 mm. The distance between the checkerboard calibration plate and the camera optical center is approximately 610 mm.
(2)
The factory-measured refractive index [27] of K9 optical glass is 1.5437, with a thickness of 20 mm, which is considered as the true value.
(3)
The number of target points is 5 × 6, with a distance of 39 mm between adjacent points.
(4)
The motion module in Figure 9 can move in the left and right directions. Before the experiment starts, the motion module can move the target near the working distance of 600 mm. The motion module of the platform does not require high motion accuracy.

4.2. Experimental Validation of Camera Imaging Model under Non-Parallel Plane Refraction

This section verifies the positioning accuracy and measurement accuracy of the camera imaging model (hereinafter referred to as the refractive imaging model) under non-parallel plane refraction. Each set of experiments kept the last target attitude fixed, and images without and with plate glass were collected separately. To compare the positioning error and measurement error under different distortion center positions, the experimental section conducted two sets of experiments with different glass postures: (a) Glass attitude 1 and (b) Glass attitude 2. Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14 show experimental data under two different glass postures.
Figure 10 shows the distribution of image deviations under non-parallel plane refraction. Arrows indicate the direction of distortion. As shown in Figure 10a, the minimum deviation of the image points is 0.5 pixels, and the maximum deviation is 9.2 pixels. As shown in Figure 10b, the minimum deviation of the image points is 2.3 pixels, and the maximum deviation is 12.1 pixels. From the figure, it is evident that the distortion center is not the main point of the camera. The distortion center is shifted because the camera’s optical axis is not parallel to the normal vector of the refraction plane of the plate glass.
In Figure 11, the cyan point is the distortion center point. The distortion centers of the two experimental groups obtained by the least squares method are (953.07, 432.1) and (92.2, 395.81), respectively. In the two experiments shown in Figure 11, due to the difference in glass placement attitude, there was a significant change in the intersection point between the normal vector passing through the optical center and the image plane. The normal vectors of the refraction plane for the two sets of experiments are n1 = (0.1268, −0.0284, 1) and n2 = (−0.2262, −0.0431, 1). The camera rotation model corrects for the non-parallelism of the camera optical axis and the normal vector of the refraction plane. Figure 12 shows the image points before and after rotational correction with glass refraction.
To verify the localization accuracy of the refraction imaging model, both the pinhole camera model and the refraction imaging model are utilized to measure and compare the points on the 2D measurement plane. As shown in Figure 13, P1 represents the pinhole camera model under glass attitude 1. R1 represents the refraction imaging model under glass attitude 1. P2 represents the pinhole camera model under glass attitude 2. R2 represents the refraction imaging model under glass attitude 2. The two experiments are shown in Figure 13. The maximum localization errors of the pinhole camera model are 2.2707 mm and 3.1139 mm, respectively. The maximum localization errors of the refracted imaging model are 0.2428 mm and 0.3394 mm, respectively. The RMSE of positioning errors using the pinhole camera model are 1.3441 mm and 1.9937 mm. The RMSE of localization errors using the refraction imaging model are 0.1135 mm and 0.1484 mm, respectively.
Then, the measurement accuracy of the refraction imaging model is verified. To comprehensively reflect the impact of plane refraction on measurement accuracy, pinhole camera models and refractive imaging models are used to measure line segments L1, L2, L3, R1, R2, and R3 on the two-dimensional measurement plane. The measured line segments are shown in Figure 14.
In Table 1, the length of the line segment measured without glass is taken as the true value. The measurements of the pinhole camera model and the refraction imaging model are compared with the true value. The measurement results of two sets of experiments are shown in Table 1 and Table 2. The RMSE of the measurement errors using the pinhole camera model are 2.3821 mm and 2.4037 mm. The RMSE of the measurement errors using the refraction imaging model are 0.2130 mm and 0.2136 mm, respectively. From the above analysis, it can be concluded that the refraction imaging model can significantly improve measurement accuracy.
In Table 2, the actual dimensions between the target points of the marker are taken as the real values. It can evaluate the actual measurement accuracy of the proposed method. The RMSE of the measurement errors using the pinhole camera model were 2.2827 mm and 2.4179 mm, while the RMSE of the measurement errors using the refraction imaging model were 0.3929 mm and 0.2627 mm, respectively. From the above analysis, it can be concluded that the refraction imaging model can significantly improve measurement accuracy.

4.3. Experiments on 3D Measurement of Binocular Vision under Plane Refraction

First, the Zhang calibration method calibrates the internal and external parameters of the two cameras. The method proposed solves the refraction plane normal vector. Table 3 demonstrates the calibration parameters. Where (fx, fy) is the camera focal length, (u0, v0) is the coordinates of the camera main point, kc is the distortion coefficient, MR is the rotation matrix, MT is the translation matrix, (uc, vc) is the distortion center, and n is the normal vector of the refraction plane of the plane glass. Find the center of distortion and the normal vector of the refraction plane. In Figure 15, the cyan point is the center distortion point for the left and right cameras.

4.3.1. Experimental Verification of Epipolar Constraint Model for Independent Refractive Plane Imaging

After calibration of the internal and external parameters of the binocular vision system is completed, the independent refraction plane imaging epipolar constraint model is experimentally validated. It has the same basic principles as the shared refraction plane imaging epipolar constraint model and has a wider range of applications. Therefore, this paper only analyzes the epipolar constraint model of independent refractive plane imaging.
In the experiment, keeping the optical axis of the left and right cameras not parallel to the normal vector of the refractive plane, the cameras collect one image with the plate glass and one image without the plate glass.
To utilize a binocular vision system for 3D measurement, it is essential to initially conduct feature point matching between the left and right images. However, Figure 16 demonstrates that the polar constraint principle of traditional binocular vision systems is rendered ineffective due to the influence of the plane refraction effect. The green line represents the limit constraint. In Figure 17, Position 1 indicates a large difference in the longitudinal tilt angle of the left and right plate glass. In Figure 17, Position 2 indicates that the left and right plate glass are parallel.
In Figure 17, when the tilt angle difference between the left and right plate glass in the longitudinal direction is large, the matching error without refraction is less than 1 pixel, and its RMSE is 0.2343 pixels. The RMSE of the matching error in the presence of refraction is 11.1107 pixels. The traditional principle of polar constraints is no longer applicable to the binocular vision systems affected by plane refraction.
But it’s not that the polar constraint principle will fail as long as there is refraction. When the left and right plate glass are nearly parallel in the longitudinal direction, the refraction has minimal impact on the coordinates v of the image points. The normal vector of the refractive surface of the left plate glass is nl = (−0.1181, −0.2283, 1). The normal vector of the refractive surface of the right plate glass is nr = (0.0063, −0.2396, 1). When the left and right plate glass are nearly parallel in the longitudinal direction, The matching error with refraction is less than 1 pixel, and the RMSE of the matching error with refraction is 0.7573 pixels. The traditional polar constraint principle remains applicable in this situation. In Figure 17, Position 1 indicates a large difference in the longitudinal tilt angle of the left and right plate glass. In Figure 17, Position 2 indicates that the left and right plate glass are parallel.

4.3.2. Experimental Validation of a Triangulation Model for Binocular Vision System under Independent Refraction Planes

Finally, the proposed triangulation model of the binocular vision system under an independent refraction plane is experimentally validated. It compares depth-of-field errors, positioning errors, and measurement errors for space points with and without considering refraction effects. Figure 18a shows the comparison of depth-of-field between the checkerboard corners before and after correction, where the depth-of-field calculated without refraction is taken as the true value. Figure 18b shows the comparison of the depth-of-field error before and after correction. The RMSE of the depth-of-field error before and after correction are 2.9902 mm and 0.3187 mm, respectively.
Figure 18b shows the comparison of positioning errors of checkerboard corner points before and after correction. The 3D coordinates calculated without refraction are used as the true spatial point coordinates. Figure 19 shows that the RMSE of the positioning errors before and after correction are 3.5661 mm and 0.3465 mm, respectively.

5. Discussion

This paper presents a complete vision measurement scheme applicable to the presence of plate glass refraction effects. The effectiveness of the scheme was verified through a series of experiments.

5.1. Analysis of the Proposed Method

In Section 3.2, the normal vector n of the refractive surface is used to describe the camera imaging model accurately when the camera’s optical axis is not parallel to the normal vector of the refractive plane. The distortion center is used to obtain the normal vector of the refraction plane of the plate glass based on the expansion offset effect of plane refraction. The established model for image distortion correction under plane refraction realizes the correction of image distortion. This model can not only eliminate image distortion but also intuitively reflect the impact of different incident angles and depth of field on measurement error.

5.2. Stability Experiment under Simulated Tunnel Construction Environments

Two types of stains shown in Figure 19 were attached to the outer surface of the plate glass to analyze how the tunnel construction environment affects the proposed plate glass calibration method. Refer to previous experiments on glass pollution. The first experiment (Non-pollution) is conducted under the ideal condition where glass is not polluted. The second experiment (Pollution A) simulated the condition of glass polluted by sewage. The third experiment (Pollution B) simulates the condition where glass is polluted by sludge and partially obstructed. According to the above experimental scheme, the calibration experiment of the refraction plane of plate glass under the simulated tunnel construction environment is carried out.
The plate glass calibration is accomplished using target points that are unpolluted or only slightly polluted. The experimental results are shown in Table 4.
According to Table 4, compared to the Non-pollution condition, the cosine similarity of the unit normal vector of the refractive surface under both polluted conditions is close to 1. This shows that pollution A and pollution B have less of an effect on the solution accuracy of the proposed method.

5.3. Future Work

Section 4.3 conducted stability experiments in simulated tunnel construction environments. However, the real tunnel construction scenes are even more harsh. Image restoration using deep learning techniques serves as a set of potential solutions. The plate glass used for the experiments in this paper is K9 optical glass. However, in practical applications, some explosion-proof tempered glass will be used, whose glass performance may not be as stable as optical glass. In the future, its practical application scenarios can be considered, and several experiments can be conducted using glass with different parameters to verify the stability of the model.

6. Conclusions

To overcome the failure of conventional vision measurement techniques under plate glass refraction, this paper provides an effective and stable solution for refraction imaging modelling, glass parameter calibration, and 3D coordinate solutions. It analytically realizes the correction of refractive distortion images. The glass normal vector is obtained from the calibration. When the camera’s optical axis is not parallel to the normal vector of the refractive plane, a camera rotation model and a camera imaging model under non-parallel plane refraction are established to correct refractive errors. In particular, an independent refractive plane triangulation model is proposed to address the issue of triangulation failure under planar refraction. This experiment verifies the positioning accuracy and measurement accuracy of the camera imaging model under non-parallel plane refraction. These experimental results show that the RMSE of the positioning errors before correction are 1.3441 mm and 1.9937 mm. The positioning errors after correction are 0.1135 mm and 0.1484 mm, respectively. The measurement errors before correction are 2.3821 mm and 2.4037 mm. The measurement errors after correction are 0.2130 mm and 0.2136 mm, respectively. This experiment verifies the accuracy of the binocular vision system triangulation model under independent refractive planes. The RMSE of the depth of field error before and after correction are 2.9902 mm and 0.3187 mm, respectively. The RMSE of the positioning errors before and after correction are 3.5661 mm and 0.3465 mm, respectively. From the above analysis, the proposed triangulation model under independent refractive planes greatly improves the accuracy of depth-of-field and positioning. This proves the effectiveness of the proposed model. While this solution is designed for tool changer robot vision systems in harsh construction environments, it can serve as a reference for any vision measurement application under glass window protection.

Author Contributions

Z.W., J.H., H.Z. and Z.F. conceived and designed the experiments; Z.W., J.H. and Z.F. performed the simulations; Z.W., F.Y. and Z.F. analyzed the results; Z.W., J.H. and H.Z. wrote the paper; and F.Y., Z.F. and S.C. technically reviewed the paper. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Natural Science Foundation of China [grant number 52275236] and the Unveiling the List and Leading Major Special Science and Technology Project of Liaoning Province [grant number 2022JH1/10400031].

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The authors confirm that the data supporting the findings of this study are available from the corresponding author, upon reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Huo, J.; Zhang, H.; Meng, Z.; Yang, F.; Yang, G. A Flexible Calibration Method Based on Small Planar Target for Defocused Cameras. Opt. Lasers Eng. 2022, 157, 107125. [Google Scholar] [CrossRef]
  2. Meng, Z.; Zhang, H.; Guo, D.; Chen, S.; Huo, J. Defocused Calibration for Large Field-of-View Binocular Cameras. Autom. Constr. 2023, 147, 104737. [Google Scholar] [CrossRef]
  3. Zhang, H.; Huo, J.; Yang, F.; Han, J.; Guo, D. A Flexible Calibration Method for Large-Range Binocular Vision System Based on State Transformation. Opt. Laser Technol. 2023, 164, 109546. [Google Scholar] [CrossRef]
  4. Shen, X.; Chen, X.; Fu, Y.; Cao, C.; Yuan, D.; Li, X.; Xiao, Y. Prediction and Analysis of Slurry Shield TBM Disc Cutter Wear and Its Application in Cutter Change Time. Wear 2022, 498–499, 204314. [Google Scholar] [CrossRef]
  5. Ngo, N.-V.; Hsu, Q.-C.; Hsiao, W.-L.; Yang, C.-J. Development of a Simple Three-Dimensional Machine-Vision Measurement System for in-Process Mechanical Parts. Adv. Mech. Eng. 2017, 9, 168781401771718. [Google Scholar] [CrossRef]
  6. Sun, Q.; Hou, Y.; Tan, Q.; Li, G. A Planar-Dimensions Machine Vision Measurement Method Based on Lens Distortion Correction. Sci. World J. 2013, 2013, 963621. [Google Scholar] [CrossRef] [PubMed]
  7. Chadebecq, F.; Vasconcelos, F.; Lacher, R.; Maneas, E.; Desjardins, A.; Ourselin, S.; Vercauteren, T.; Stoyanov, D. Refractive Two-View Reconstruction for Underwater 3D Vision. Int. J. Comput. Vis. 2020, 128, 1101–1117. [Google Scholar] [CrossRef] [PubMed]
  8. Rouxel, T.; Ji, H.; Hammouda, T.; Moréac, A. Poisson’s Ratio and the Densification of Glass under High Pressure. Phys. Rev. Lett. 2008, 100, 225501. [Google Scholar] [CrossRef] [PubMed]
  9. Elnashef, B.; Filin, S. A Three-Point Solution with Scale Estimation Ability for Two-View Plate-Refractive Underwater Photogrammetry. ISPRS J. Photogramm. Remote Sens. 2023, 198, 223–237. [Google Scholar] [CrossRef]
  10. Kwon, Y.; Casebolt, J.B. Effects of Light Refraction on the Accuracy of Camera Calibration and Reconstruction in Underwater Motion Analysis. Sports Biomech. 2006, 5, 95–120. [Google Scholar] [CrossRef] [PubMed]
  11. Sanchez-Ferreira, C.; Mori, J.Y.; Llanos, C.H.; Fortaleza, E. Development of a Stereo Vision Measurement Architecture for an Underwater Robot. In Proceedings of the 2013 IEEE 4th Latin American Symposium on Circuits and Systems (LASCAS), Cusco, Peru, 27 February–1 March 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 1–4. [Google Scholar]
  12. Treibitz, T.; Schechner, Y.; Kunz, C.; Singh, H. Plate Refractive Geometry. IEEE Trans. Pattern Anal. Mach. Intell. 2012, 34, 51–65. [Google Scholar] [CrossRef]
  13. Agrawal, A.; Ramalingam, S.; Taguchi, Y.; Chari, V. A Theory of Multi-Layer Plate Refractive Geometry. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 16–21 June 2012; IEEE: Piscataway, NJ, USA, 2012; pp. 3346–3353. [Google Scholar]
  14. Chen, X.; Yang, Y.-H. Two-View Camera Housing Parameters Calibration for Multi-Layer Plate Refractive Interface. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; IEEE: Piscataway, NJ, USA, 2014; pp. 524–531. [Google Scholar]
  15. Shimizu, M.; Okutomi, M. Reflection Stereo—Novel Monocular Stereo Using a Transparent Plate -. In Proceedings of the 3rd Canadian Conference on Computer and Robot Vision (CRV’06), Quebec City, QC, Canada, 7–9 June 2006; IEEE: Piscataway, NJ, USA, 2006; p. 14. [Google Scholar]
  16. Shimizu, M.; Okutomi, M. Monocular Range Estimation through a Double-Sided Half-Mirror Plate. In Proceedings of the Fourth Canadian Conference on Computer and Robot Vision (CRV ’07), Montreal, QC, Canada, 28–30 May 2007; IEEE: Piscataway, NJ, USA, 2007; pp. 347–354. [Google Scholar]
  17. Gong, Z.; Liu, Z.; Zhang, G. Flexible Method of Refraction Correction in Vision Measurement Systems with Multiple Glass Ports. Opt. Express 2017, 25, 831. [Google Scholar] [CrossRef] [PubMed]
  18. Ke, X.; Sutton, M.A.; Lessner, S.M.; Yost, M. Robust Stereo Vision and Calibration Methodology for Accurate Three-Dimensional Digital Image Correlation Measurements on Submerged Objects. J. Strain Anal. Eng. Des. 2008, 43, 689–704. [Google Scholar] [CrossRef]
  19. Huang, S.; Feng, M.C.; Zheng, T.X.; Li, F.; Wang, J.Q.; Xiao, L.F. A Novel Multi-Camera Calibration Method Based on Plate Refractive Geometry. IOP Conf. Ser. Mater. Sci. Eng. 2018, 320, 012016. [Google Scholar] [CrossRef]
  20. Shortis, M. Calibration Techniques for Accurate Measurements by Underwater Camera Systems. Sensors 2015, 15, 30810–30826. [Google Scholar] [CrossRef] [PubMed]
  21. Yamashita, A.; Fujii, A.; Kaneko, T. Three Dimensional Measurement of Objects in Liquid and Estimation of Refractive Index of Liquid by Using Images of Water Surface with a Stereo Vision System. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, 19–23 May 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 974–979. [Google Scholar]
  22. Huang, L.; Zhao, X.; Cai, S.; Liu, Y. Plate Refractive Camera Model and Its Applications. J. Electron. Imaging 2017, 26, 023020. [Google Scholar] [CrossRef]
  23. Gedge, J.; Gong, M.; Yang, Y.-H. Refractive Epipolar Geometry for Underwater Stereo Matching. In Proceedings of the 2011 Canadian Conference on Computer and Robot Vision, St. Johns, NL, Canada, 25–27 May 2011; IEEE: Piscataway, NJ, USA, 2011; pp. 146–152. [Google Scholar]
  24. Zhang, Z. A Flexible New Technique for Camera Calibration. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 1330–1334. [Google Scholar] [CrossRef]
  25. Zhang, Z. Flexible Camera Calibration by Viewing a Plane from Unknown Orientations. In Proceedings of the Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 20–25 September 1999; IEEE: Piscataway, NJ, USA, 1999; Volume 1, pp. 666–673. [Google Scholar]
  26. Zhang, H.; Huo, J.; Meng, Z.; Wu, Z.; Ma, Y. Imaging Model and Vision Measurement Method Considering Refraction Effect of Plate Glass Windows in Tunnel Construction. Opt. Lasers Eng. 2023, 170, 107782. [Google Scholar] [CrossRef]
  27. Wen, Z.; Curran, J.M.; Harbison, S.-A.; Wevers, G. Bayesian mixture modelling for glass refractive index measurement. Sci. Justice 2021, 61, 345–355. [Google Scholar] [CrossRef] [PubMed]
Sensors 24 00066 g001
Figure 1. The TBM diagram of the changing area.
Figure 1. The TBM diagram of the changing area.
Sensors 24 00066 g001
Sensors 24 00066 g002
Figure 2. The basic content of this article.
Figure 2. The basic content of this article.
Sensors 24 00066 g002
Sensors 24 00066 g003
Figure 3. An Imaging model when the camera’s optical axis is not parallel to the normal vector of the refraction plane. (a) Plane refraction imaging and (b) image points on the imaging surface.
Figure 3. An Imaging model when the camera’s optical axis is not parallel to the normal vector of the refraction plane. (a) Plane refraction imaging and (b) image points on the imaging surface.
Sensors 24 00066 g003
Sensors 24 00066 g004
Figure 4. Non-parallel plane refractive camera rotation model.
Figure 4. Non-parallel plane refractive camera rotation model.
Sensors 24 00066 g004
Sensors 24 00066 g005
Figure 5. Non-parallel plane refractive camera imaging model.
Figure 5. Non-parallel plane refractive camera imaging model.
Sensors 24 00066 g005
Sensors 24 00066 g006
Figure 6. Independent refractive surface (a) imaging model and (b) epipolar constraint model.
Figure 6. Independent refractive surface (a) imaging model and (b) epipolar constraint model.
Sensors 24 00066 g006
Sensors 24 00066 g007
Figure 7. Triangulation measurement model under an independent refraction plane.
Figure 7. Triangulation measurement model under an independent refraction plane.
Sensors 24 00066 g007
Sensors 24 00066 g008
Figure 8. Overall flow chart of binocular measurement under plate refraction.
Figure 8. Overall flow chart of binocular measurement under plate refraction.
Sensors 24 00066 g008
Sensors 24 00066 g009
Figure 9. Experimental platform for vision program of the tool-change robot.
Figure 9. Experimental platform for vision program of the tool-change robot.
Sensors 24 00066 g009
Sensors 24 00066 g010
Figure 10. Image deviation distribution plot. (a) Glass attitude 1 and (b) glass attitude 2.
Figure 10. Image deviation distribution plot. (a) Glass attitude 1 and (b) glass attitude 2.
Sensors 24 00066 g010
Sensors 24 00066 g011
Figure 11. Determination of the distortion center. (a) Glass attitude 1 and (b) glass attitude 2.
Figure 11. Determination of the distortion center. (a) Glass attitude 1 and (b) glass attitude 2.
Sensors 24 00066 g011
Sensors 24 00066 g012
Figure 12. Image point comparison before and after camera rotation with glass. (a) Glass attitude 1 and (b) glass attitude 2.
Figure 12. Image point comparison before and after camera rotation with glass. (a) Glass attitude 1 and (b) glass attitude 2.
Sensors 24 00066 g012
Sensors 24 00066 g013
Figure 13. Pinhole camera model versus refractive imaging model positioning error.
Figure 13. Pinhole camera model versus refractive imaging model positioning error.
Sensors 24 00066 g013
Sensors 24 00066 g014
Figure 14. The measured segment position.
Figure 14. The measured segment position.
Sensors 24 00066 g014
Sensors 24 00066 g015
Figure 15. Determination of the distortion center. (a) Left camera and (b) right camera.
Figure 15. Determination of the distortion center. (a) Left camera and (b) right camera.
Sensors 24 00066 g015
Sensors 24 00066 g016
Figure 16. Epipolar correction diagram with glass.
Figure 16. Epipolar correction diagram with glass.
Sensors 24 00066 g016
Sensors 24 00066 g017
Figure 17. Matching error with and without refraction at different glass positions.
Figure 17. Matching error with and without refraction at different glass positions.
Sensors 24 00066 g017
Sensors 24 00066 g018
Figure 18. Comparison before and after calibration. (a) Depth of field and (b) depth of field error and positioning error.
Figure 18. Comparison before and after calibration. (a) Depth of field and (b) depth of field error and positioning error.
Sensors 24 00066 g018
Sensors 24 00066 g019
Figure 19. Pollution type diagram. (a) Glass is polluted by sewage. (b) Glass is polluted by sludge.
Figure 19. Pollution type diagram. (a) Glass is polluted by sewage. (b) Glass is polluted by sludge.
Sensors 24 00066 g019
Table 1. Measurement error in Experiment 1. Units: mm.
Table 1. Measurement error in Experiment 1. Units: mm.
GroupLine SegmentReal ValuePinhole Camera ModelRefraction Imaging ModelPinhole Camera Model Measurement ErrorRefraction Imaging Model Measurement Errors
experiment oneL1156.6668158.5312156.86911.86450.2023
L2194.8575197.1360195.08962.27850.2321
L3249.9980252.8859250.23252.89780.2445
R1155.6221157.3648155.66321.74270.0411
R2194.9240197.2314195.14392.30740.2199
R3249.0547251.9907249.31452.93600.2598
experiment twoL1156.5188158.2746156.44721.75580.0716
L2194.7974197.1148195.06832.31750.2710
L3249.9564252.8827250.13992.92630.1835
R1155.7484157.6053155.92551.85690.1770
R2194.8289197.1602195.06122.33130.2323
R3249.9526252.9136250.22882.96100.2762
Table 2. Measurement error in Experiment 2. Units: mm.
Table 2. Measurement error in Experiment 2. Units: mm.
GroupLine SegmentReal ValuePinhole Camera ModelRefraction Imaging ModelPinhole Camera Model Measurement ErrorRefraction Imaging Model Measurement Errors
experiment oneL1156.0000158.5312156.86912.53120.8691
L2195.0000197.1360195.08962.13600.0896
L3249.7218252.8859250.23253.16410.5107
R1156.0000157.3648155.66321.36480.3368
R2195.0000197.2314195.14392.23140.1439
R3249.7218251.9907249.31452.26890.4073
experiment twoL1156.0000158.2746156.44722.27460.4472
L2195.0000197.1148195.06832.11480.0683
L3249.7218252.8827250.13993.16090.4181
R1156.0000157.6053155.92551.60530.0745
R2195.0000197.1602195.06122.16020.0612
R3249.7218252.9136250.22883.19180.5070
Table 3. Camera parameters.
Table 3. Camera parameters.
Left CameraRight Camera
(fx, fy)/mm(2438.07, 2438.08)(2443.13, 2443.13)
(u0, v0)/pixel(643.98, 500.01)(647.83, 487.49)
kc(−0.4082, 0.4359)(−0.4100, 0.4389)
MR 0.9572 0.0008 − 0.2893 − 0.0087 0.9996 − 0.0262 0.2892 0.0275 0.9569
MT[−211.4756, 1.3306, 53.3775]T
(uc, vc)/pixel(38.21, 735.40)(1092.19, −196.90)
n(−0.2483, 0.0961, 1)(0.1823, −0.2809, 1)
Table 4. Unit normal vectors and refractive indices.
Table 4. Unit normal vectors and refractive indices.
Serial NumberUnit Normal VectorCosine Similarity
Left
glass		Non-pollution[−0.0500, −0.0475, 0.9976]T/
Pollution A[−0.0478, −0.0466, 0.9973]T0.9999972
Pollution B[−0.0475, −0.0463, 0.9977]T0.9999961
Right
glass		Non-pollution[−0.0036, −0.0235, 1.0000]T/
Pollution A[−0.0040, −0.0238, 0.9998]T0.9999999
Pollution B[−0.0043, −0.0212, 1.0006]t0.9999972
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wu, Z.; Huo, J.; Zhang, H.; Yang, F.; Chen, S.; Feng, Z. Vision Measurement Method Based on Plate Glass Window Refraction Model in Tunnel Construction. Sensors 2024, 24, 66. https://doi.org/10.3390/s24010066

AMA Style

Wu Z, Huo J, Zhang H, Yang F, Chen S, Feng Z. Vision Measurement Method Based on Plate Glass Window Refraction Model in Tunnel Construction. Sensors. 2024; 24(1):66. https://doi.org/10.3390/s24010066

Chicago/Turabian Style

Wu, Zhen, Junzhou Huo, Haidong Zhang, Fan Yang, Shangqi Chen, and Zhihao Feng. 2024. "Vision Measurement Method Based on Plate Glass Window Refraction Model in Tunnel Construction" Sensors 24, no. 1: 66. https://doi.org/10.3390/s24010066

APA Style

Wu, Z., Huo, J., Zhang, H., Yang, F., Chen, S., & Feng, Z. (2024). Vision Measurement Method Based on Plate Glass Window Refraction Model in Tunnel Construction. Sensors, 24(1), 66. https://doi.org/10.3390/s24010066

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop