CN114399631A - A 3D reconstruction and sludge identification method for the interior of a large crude oil tank - Google Patents

Abstract

The invention discloses a three-dimensional reconstruction and sludge identification method inside a large crude oil tank, comprising: acquiring a plurality of pre-processed internal images of the large crude oil tank; extracting feature points of the internal images of the large crude oil tank; The matching technology finds the position of the pixel point corresponding to the feature point in another frame of the internal image of the crude oil tank obtained after the camera moves; calculates the position of the camera according to the position of the feature point pair after the matching of the two images. Perform depth estimation of pixel points according to the pose of the camera, and restore the three-dimensional space coordinates of the pixel points; convert and splicing the three-dimensional space coordinates to obtain a point cloud map, and perform a triangular mesh on the point cloud map The 3D reconstruction model of the inside of the oil storage tank is obtained by dense reconstruction and pasting the texture. The invention can stably and efficiently display three-dimensional visualization of the visible surface inside the oil storage tank in real time, and accurately identify the position of the oil sludge, so as to clean the oil sludge on the tank body accurately and efficiently.

Description

A 3D reconstruction and sludge identification method for the interior of a large crude oil tank

technical field

The invention relates to the field of petrochemical industry, in particular to a three-dimensional reconstruction and sludge identification method inside a large crude oil tank.

Background technique

At present, 3D reconstruction has been widely used in various aspects such as autonomous driving, robotics, AR, industrial manufacturing, etc.; target detection is to find the specific location of the target in a given image, video or scene, and it is also widely used. Oil is an important energy source after coal in my country. At present, the country needs to store a large amount of crude oil and has a large number of crude oil storage tanks. However, the long-term storage of crude oil in the tank will deposit and form sludge, which will cause certain corrosion to the oil storage tank. Therefore, regular cleaning is required. In order to clearly understand the internal situation of the oil storage tank and accurately clean the oil stains, a three-dimensional reconstruction of the oil storage tank and the identification of the oil sludge are proposed.

At present, no one has done 3D reconstruction after acquiring images with a monocular infrared camera for this specific scene inside a large crude oil tank. At the same time, for the identification of sludge in the tank, the mainstream algorithm uses the traditional Canny detector for edge detection to extract the position of sludge. However, since the inside of the oil storage tank is completely dark and closed, the image quality of the inside of the oil storage tank captured by the infrared camera will be degraded; at the same time, the traditional Canny edge detection operator is sensitive to image noise, has false edges and lacks self-adaptation If it is applied to such a complex scene in an oil storage tank, the detection effect will be greatly reduced, and false detection will occur. Therefore, more accurate and efficient sludge identification devices and methods in large oil tanks are required.

SUMMARY OF THE INVENTION

The invention provides a three-dimensional reconstruction and sludge identification method inside a large crude oil tank, which is used to solve the problem that the three-dimensional reconstruction of the interior of the large crude oil tank by unmanned persons and the image quality of the interior of the oil storage tank captured by an infrared camera will decrease, and the position of the oil sludge will be extracted. The problem of greatly reduced effect. In order to solve the above technical problems, the present invention provides a three-dimensional reconstruction and sludge identification method inside a crude oil tank, comprising the following steps:

Obtain pre-processed internal images of multiple crude oil tanks;

extracting feature points of the internal image of the crude oil tank;

Using epipolar search and block matching technology to find the position of the pixel corresponding to the feature point in another frame of the internal image of the crude oil tank captured after the camera moves;

Calculate the pose of the camera according to the position of the pair of feature points that are matched between the two images captured before and after the camera moves;

Perform depth estimation of pixel points according to the pose of the camera, and restore the three-dimensional space coordinates of the pixel points according to the depth estimation;

The three-dimensional space coordinates are converted into point clouds for splicing to obtain a point cloud map, and the point cloud map is densely reconstructed by triangulation and pasted with textures to obtain a visualized three-dimensional reconstruction model inside the oil storage tank.

Preferably, after acquiring the pre-processed images of the interior of the large crude oil tank, the following steps are further included:

Using fast guided filtering to perform edge-preserving noise reduction on the internal image of the crude oil tank to obtain a noise-reduced image;

Calculate the gradient magnitude and direction of the denoised image to estimate edge strength and direction at each point;

performing non-maximum suppression according to the gradient magnitude and direction;

Determine and connect image edges by adaptively selecting thresholds through OTSU maximum inter-class variance method;

Convert and output the image sequence containing the outline of the object, and automatically identify whether the image sequence contains sludge.

Preferably, before the step of acquiring the preprocessed internal images of multiple crude oil tanks, the following steps are further included:

Receive image sequences collected by monitoring equipment;

Non-linear transformation and multi-scale convolution calculation are used to improve the brightness and contrast of the image sequence, and the preprocessed internal images of multiple crude oil tanks are obtained.

Preferably, the specific method for extracting the feature points of the internal image of the crude oil tank is:

The feature points are extracted by ORB feature extraction algorithm.

Preferably, the specific steps of calculating the pose of the camera according to the position of the pair of feature points that are matched between the two images captured before and after the camera moves are as follows:

其中E为本质矩阵，K为相机的内参矩阵，p1、p2为两张图像上匹配的特征点对；According to the matching feature point pairs on the two images, the essential matrix can be solved by using the eight-point method, and the essential matrix satisfies the formula:

Among them, E is the essential matrix, K is the internal parameter matrix of the camera, and p1 and p2 are the matching feature point pairs on the two images;

According to the following formula: E=t ^Λ R, according to the singular value decomposition, the translation vector t and rotation vector R of the camera are solved.

Preferably, the depth estimation of the pixel point is performed according to the pose of the camera, and the specific steps of restoring the three-dimensional space coordinates of the pixel point according to the depth estimation are as follows:

S151: Assuming that the depth d of a certain pixel follows a Gaussian distribution, the formula is as follows:

P(d)=N(μ,σ ² );

S152: when new data is generated, assuming that it also obeys a Gaussian distribution, the position of the projection point of a certain pixel in the reference frame at the current frame can be determined according to epipolar search and block matching;

S153: Calculate the depth and uncertainty after triangulation according to the geometric relationship;

S154: Integrate the current observation into the last estimation, and stop the calculation if the uncertainty is smaller than the set threshold, otherwise return to step S152.

The beneficial effects of the present invention are: the visible surface inside the large crude oil tank can be displayed in real time, stably and efficiently in three-dimensional visualization, and the sludge identification algorithm can be used to accurately identify the position of the sludge, so that the cleaning gun can accurately and efficiently clean the sludge on the tank. Oil stains.

Description of drawings

Fig. 1 is the flow chart of 3D reconstruction;

Fig. 2 is the flow chart of the sludge identification algorithm;

Fig. 3 is the flow chart of the key point direction of the ORB feature extraction algorithm;

Figure 4 is a flowchart of camera pose estimation;

FIG. 5 is a flowchart of restoring the three-dimensional space coordinates of a pixel point.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1

Please refer to accompanying drawing 1, a kind of crude oil tank interior three-dimensional reconstruction and sludge identification method, comprises the following steps:

S11: Acquire the pre-processed internal images of multiple crude oil tanks;

S12: extracting feature points of the internal image of the crude oil tank;

S13: Use epipolar search and block matching technology to find the position of the pixel corresponding to the feature point in another frame of the internal image of the crude oil tank captured after the camera moves;

S14: Calculate the pose of the camera according to the positions of the pair of feature points that are matched between the two images captured before and after the camera moves;

S15: Perform depth estimation of the pixel points according to the pose of the camera, and restore the three-dimensional space coordinates of the pixel points according to the depth estimation;

S16: Convert the three-dimensional space coordinates into point clouds for splicing to obtain a point cloud map, and use triangulation to densely reconstruct and paste textures on the point cloud map to obtain a visualized three-dimensional reconstruction model inside the oil storage tank.

As a preferred embodiment, as shown in FIG. 2 , after acquiring the images of the interiors of multiple crude oil tanks after preprocessing, the following steps are further included:

S21: use fast guided filtering to perform edge-preserving noise reduction on the internal image of the crude oil tank to obtain a noise-reduced image;

S22: Calculate the gradient magnitude and direction of the denoised image to estimate the edge strength and direction at each point;

S23: perform non-maximum suppression according to the gradient magnitude and direction;

S24: Determine and connect the image edges by adaptively selecting a threshold through the OTSU maximum inter-class variance method;

S25: Convert and output an image sequence containing the outline of the object, and automatically identify whether the image sequence contains sludge.

As a preferred embodiment, before the step of acquiring the pre-processed images of the interior of a plurality of crude oil tanks further includes the following steps:

S01: Receive the image sequence collected by the monitoring device;

S02: Use nonlinear transformation and multi-scale convolution calculation to improve the brightness and contrast of the image sequence, and obtain multiple pre-processed internal images of crude oil tanks.

As a preferred embodiment, the specific method for extracting the feature points of the internal image of the crude oil tank is:

The feature point extraction is realized by using the ORB feature extraction algorithm. The ORB feature extraction algorithm includes a scale invariant description and a rotation invariant description; the scale invariant description is constructed from the image sequence input to the system. Image pyramid, and in the pyramid It is realized by setting key points on each layer; the rotation invariance description is described by the ORB feature detection and vector creation algorithm to assign a direction to each key point according to the change of the gray value around the key point, and the geometric center points to The gray centroid is the direction of the key point, as shown in Figure 3, the calculation steps of the direction of the key point are as follows:

S121: In an image block B, the moment defining the image block is:

S122: The centroid of the image patch can be found by the moment:

S123: Connect the geometric center O and the centroid C of the image block to obtain a direction vector OC, so the direction of the feature point can be defined as:

θ=arctan(m ₀₁ /m ₁₀ ).

As a preferred embodiment, as shown in FIG. 4 , the specific steps for calculating the pose of the camera according to the positions of the pair of feature points that are matched between the two images captured before and after the camera moves are as follows:

其中E为本质矩阵，K为相机的内参矩阵，p1、p2为两张图像上匹配的特征点对；S141: According to the matched feature point pairs on the two images, an essential matrix can be solved by using the eight-point method, and the essential matrix satisfies the formula:

Among them, E is the essential matrix, K is the internal parameter matrix of the camera, and p1 and p2 are the matching feature point pairs on the two images;

S142: According to the following formula: E=t ^Λ R, according to singular value decomposition, solve the translation vector t and rotation vector R of the camera.

As a preferred embodiment, as shown in FIG. 5 , the depth estimation of the pixel points is performed according to the pose of the camera, and the specific steps for restoring the three-dimensional space coordinates of the pixel points according to the depth estimation are as follows:

S151: Assuming that the depth d of a certain pixel follows a Gaussian distribution, the formula is as follows:

P(d)=N(μ,σ ² );

S152: when new data is generated, assuming that it also obeys a Gaussian distribution, the position of the projection point of a certain pixel in the reference frame at the current frame can be determined according to epipolar search and block matching;

S153: Calculate the depth and uncertainty after triangulation according to the geometric relationship;

S154: Integrate the current observation into the last estimation, and stop the calculation if the uncertainty is smaller than the set threshold, otherwise return to step S152.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, and substitutions can be made in these embodiments without departing from the principle and spirit of the invention and modifications, the scope of the invention is defined by the appended claims and their equivalents.

Claims (6)

1. A three-dimensional reconstruction and oil sludge identification method for the interior of a crude oil large tank is characterized by comprising the following steps:

acquiring images of the interior of a plurality of crude oil large tanks after pretreatment;

extracting characteristic points of the image in the crude oil tank;

searching the position of a pixel point corresponding to the characteristic point in the image in the other frame of crude oil tank obtained by shooting after the camera moves by adopting an epipolar line search and block matching technology;

calculating the pose of the camera according to the positions of the feature point pairs which are matched with the images shot before and after the movement of the two cameras;

performing depth estimation on pixel points according to the pose of the camera, and recovering three-dimensional space coordinates of the pixel points according to the depth estimation;

converting the three-dimensional space coordinates into point clouds for splicing to obtain a point cloud map, and performing triangular meshing dense reconstruction and texture pasting on the point cloud map to obtain a visual three-dimensional reconstruction model of the interior of the oil storage tank.

2. The method for three-dimensional reconstruction and sludge identification of the interior of the large crude oil tank as claimed in claim 1, wherein after the pre-processed images of the interior of the large crude oil tank are obtained, the method further comprises the following steps:

performing edge protection and noise reduction on the image in the crude oil tank by adopting rapid guide filtering to obtain a noise reduction image;

calculating the gradient amplitude and direction of the noise-reduced image to estimate the edge strength and direction at each point;

carrying out non-maximum suppression according to the gradient amplitude and direction;

adaptively selecting a threshold value by an OTSU maximum inter-class variance method to determine and connect image edges;

and converting and outputting an image sequence containing the object contour line, and automatically identifying whether the image sequence contains oil sludge.

3. The method of claim 1, wherein the step of obtaining the preprocessed images of the interior of the crude oil tank further comprises the following steps:

receiving an image sequence acquired by monitoring equipment;

and adopting nonlinear transformation and multi-scale convolution calculation to improve the brightness and contrast of the image sequence to obtain the preprocessed images of the interior of the large crude oil tanks.

4. The method for three-dimensional reconstruction and oil sludge identification of the interior of the large crude oil tank as claimed in claim 1, wherein the specific method for extracting the feature points of the image of the interior of the large crude oil tank is as follows:

and (4) extracting the feature points by adopting an ORB feature extraction algorithm.

5. The method for three-dimensional reconstruction and oil sludge identification of the interior of the crude oil large tank according to claim 1, wherein the specific steps of calculating the pose of the camera according to the positions of the feature point pairs which are matched and completed by the images shot before and after the movement of the two cameras are as follows:

according to the matched characteristic point pairs on the two images, an essential matrix can be solved by using an eight-point method, wherein the essential matrix meets the formula:

wherein E is an essential matrix, K is an internal reference matrix of the camera, and p1 and p2 are feature point pairs matched on the two images;

the following formula: e ═ t^ΛAnd R, solving the translation vector t and the rotation vector R of the camera according to singular value decomposition.

6. The method for three-dimensional reconstruction and oil sludge identification in the crude oil large tank according to claim 1 is characterized in that depth estimation of pixel points is performed according to the pose of the camera, and the specific steps of recovering three-dimensional space coordinates of the pixel points according to the depth estimation are as follows:

s151: assuming that the depth d of a certain pixel follows gaussian distribution, the formula is as follows:

P(d)＝N(μ，σ²)；

s152: when new data is generated, assuming that the new data also obeys Gaussian distribution, the position of a certain pixel in the reference frame at the projection point of the current frame can be determined according to epipolar line search and block matching;

s153: calculating the depth and uncertainty of the triangulated depth according to the geometric relationship;

s154: and fusing the current observation into the last estimation, if the uncertainty is smaller than the set threshold, stopping the calculation, otherwise, returning to the step S152.

Images