CN111932642B - Method, device and equipment for measuring and calculating volume of structural crack and storage medium - Google Patents
Method, device and equipment for measuring and calculating volume of structural crack and storage medium Download PDFInfo
- Publication number
- CN111932642B CN111932642B CN202011033662.0A CN202011033662A CN111932642B CN 111932642 B CN111932642 B CN 111932642B CN 202011033662 A CN202011033662 A CN 202011033662A CN 111932642 B CN111932642 B CN 111932642B
- Authority
- CN
- China
- Prior art keywords
- crack
- volume
- calculating
- structural
- measuring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000004364 calculation method Methods 0.000 claims abstract description 39
- 239000000463 material Substances 0.000 claims abstract description 19
- 238000012795 verification Methods 0.000 claims description 70
- 238000005259 measurement Methods 0.000 claims description 48
- 238000012549 training Methods 0.000 claims description 27
- 238000010276 construction Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012360 testing method Methods 0.000 claims description 6
- 238000001514 detection method Methods 0.000 abstract description 18
- 230000002787 reinforcement Effects 0.000 abstract description 4
- 230000008439 repair process Effects 0.000 abstract description 4
- 239000004567 concrete Substances 0.000 description 13
- 230000008569 process Effects 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 6
- 206010063385 Intellectualisation Diseases 0.000 description 3
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 238000013526 transfer learning Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000011150 reinforced concrete Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Analysis (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Computational Linguistics (AREA)
- Computational Mathematics (AREA)
- Pathology (AREA)
- Immunology (AREA)
- Biochemistry (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Signal Processing (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Geometry (AREA)
- Quality & Reliability (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The application discloses a method, a device, equipment and a storage medium for measuring and calculating the volume of a structural crack, wherein the method comprises the following steps: acquiring a plurality of crack sample images through a calibrated and registered depth camera and a monocular camera, and calculating corresponding crack volume values in the crack sample images; constructing a structural crack volume measuring and calculating model by taking the collected crack sample image as input and the calculated crack volume value as output; acquiring a crack RGB image of the structure to be detected through a monocular camera; and inputting the collected RGB image of the crack of the structure to be measured into the constructed structure crack volume measuring and calculating model for calculation to obtain the volume measuring and calculating information of the crack of the structure to be measured. The method and the device can greatly improve the detection precision and the detection efficiency, are suitable for measuring and calculating the crack volumes of various material structures, have strong universality, can realize real-time non-contact detection, have low cost and high precision, are easy to operate, and provide effective ways for damage repair, structure reinforcement and the like of the structure.
Description
Technical Field
The invention relates to the technical field of engineering structure crack detection, in particular to a method, a device, equipment and a storage medium for measuring and calculating the volume of a structure crack.
Background
With the rapid development of construction industry, large-volume reinforced concrete structures in high-rise building structures and underground construction projects are more and more, but the property of the concrete structures that cracks are easily generated in the construction process brings great troubles to engineering design and constructors, and meanwhile, serious potential quality hazards are buried in the engineering structures.
In the research on the construction of mass concrete structures, the current mass concrete is still in the starting development stage, and the exploration and deepening process is still carried out on the structural characteristics, the material characteristics and the construction process requirements of the mass concrete, particularly on the prevention and treatment of the mass concrete cracks.
Therefore, in order to ensure the construction quality of the mass concrete, how to prevent and control the crack defect of the mass concrete, and solve the problem that the volume of the crack of the mass concrete structure is difficult to calculate by engineering cost workers, technical problems to be solved by the technical personnel in the field are needed.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus, a device and a storage medium for measuring and calculating a structural crack volume, which have the advantages of high detection accuracy, high detection efficiency, simple operation, low cost and no damage to a structural structure. The specific scheme is as follows:
a method for measuring and calculating the volume of a structural crack comprises the following steps:
acquiring a plurality of crack sample images through a calibrated and registered depth camera and a monocular camera, and calculating corresponding crack volume values in the crack sample images;
constructing a structural crack volume measuring and calculating model by taking the collected crack sample image as input and the calculated crack volume value as output;
acquiring a crack RGB image of the structure to be detected through a monocular camera;
and inputting the collected RGB image of the crack of the structure to be measured into the constructed crack volume measuring and calculating model of the structure to be measured for calculation, so as to obtain the volume measuring and calculating information of the crack of the structure to be measured.
Preferably, in the method for measuring and calculating the structural crack volume provided in the embodiment of the present invention, calculating a corresponding crack volume value in each crack sample image specifically includes:
acquiring the area of the cross section of the crack according to the number of pixels in a polygon formed by the cross section of the crack in an RGB image obtained by a monocular camera;
and integrating to obtain the corresponding crack volume value in each crack sample image according to the crack depth image obtained by the depth camera and the obtained crack cross section area.
Preferably, in the method for measuring and calculating the structural crack volume provided by the embodiment of the present invention, the crack volume value corresponding to each crack sample image is obtained by integrating using the following formula:
wherein,
the volume value of the crack is taken as the value of the volume of the crack,
is as followsiThe area of the cross section of each crack; performing crack depth map obtained by the depth camera in the vertical directionhAre equally divided, each infinitesimal being
；
Is a scale factor of the crack depth map in the vertical direction to the actual size.
Preferably, in the method for measuring and calculating the volume of the structural crack provided in the embodiment of the present invention, the constructing a structural crack volume measuring and calculating model specifically includes:
training a Faster-RCNN model by utilizing a training sample set to obtain a structural crack volume measuring and calculating model; the training sample set at least comprises a plurality of crack sample images which are obtained by carrying out field actual measurement, model test or finite element calculation on the same material as the structural material to be tested.
Preferably, in the method for measuring and calculating the crack volume of the structure provided by the embodiment of the present invention, before inputting the acquired RGB image of the crack of the structure to be measured into the constructed structure crack volume measuring and calculating model for calculation, the method further includes:
obtaining a model verification dataset; the model verification data set comprises a plurality of verification sample images, and each verification sample image is calibrated with a corresponding crack depth map in advance and is a structural crack image which is the same as the structural material to be detected;
inputting each verification sample image into the constructed structural crack volume measurement model to obtain structural crack volume measurement information of each verification sample image;
calculating the accuracy of the structural crack volume calculation model according to the structural crack volume calculation information, the known actual crack volume information and the total number of the verification sample images of each verification sample image;
judging whether the accuracy of the structural crack volume measuring and calculating model is smaller than a preset threshold value or not;
if so, increasing the crack sample images in the training sample set, and retraining the structural crack volume measuring and calculating model until the accuracy is greater than or equal to the preset threshold; and if not, using the structural crack volume measuring and calculating model for subsequently measuring and calculating the volume information of the structural crack to be measured.
Preferably, in the method for measuring and calculating the structural crack volume provided in the embodiment of the present invention, the calculating the accuracy of the structural crack volume measurement model according to the structural crack volume measurement information, the known actual crack volume information of each verified sample image and the total number of the verified sample images specifically includes:
counting the qualified number of the verification sample images of which the difference value between the structure crack measurement information and the known actual crack volume information of each verification sample image is less than or equal to a preset deviation value;
and calculating the ratio of the qualified number and the total number of the verification sample images to be used as the accuracy of the structural crack volume measurement model.
The embodiment of the invention also provides a device for measuring and calculating the volume of the structural crack, which comprises:
the sample data acquisition module is used for acquiring a plurality of crack sample images through the calibrated and registered depth camera and the monocular camera and calculating corresponding crack volume values in the crack sample images;
the model pre-construction module is used for constructing a structural crack volume measurement model by taking the collected crack sample image as input and the calculated crack volume value as output;
the to-be-detected image acquisition module is used for acquiring a crack RGB image of the to-be-detected structure through a monocular camera;
and the crack volume measuring and calculating module is used for inputting the acquired RGB image of the crack of the structure to be measured into the constructed crack volume measuring and calculating model of the structure to be measured for calculation so as to obtain the volume measuring and calculating information of the crack of the structure to be measured.
Preferably, in the above apparatus for measuring and calculating volume of structural crack provided in an embodiment of the present invention, the apparatus further includes: the model verification module is used for verifying the structural crack volume measurement and calculation model; the model verification module includes:
the model verification data set acquisition sub-module is used for acquiring a model verification data set; the model verification data set comprises a plurality of verification sample images, and each verification sample image is calibrated with a corresponding crack depth map in advance and is a structural crack image which is the same as the structural material to be detected;
the crack volume measuring and calculating submodule is used for inputting each verification sample image into the constructed structural crack volume measuring and calculating model to obtain structural crack volume measuring and calculating information of each verification sample image;
the accuracy calculation submodule is used for calculating the accuracy of the structural crack volume measuring and calculating model according to the structural crack volume measuring and calculating information of each verified sample image, the known actual crack volume information and the total number of the verified sample images;
and the model retraining submodule is used for increasing the crack sample images in the training sample set when the accuracy of the structural crack volume measuring and calculating model is smaller than a preset threshold value, and retraining the structural crack volume measuring and calculating model until the accuracy is larger than or equal to the preset threshold value.
The embodiment of the present invention further provides a device for measuring and calculating the volume of a structural crack, which includes a processor and a memory, wherein the processor implements the method for measuring and calculating the volume of the structural crack provided by the embodiment of the present invention when executing the computer program stored in the memory.
The embodiment of the present invention further provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the method for measuring and calculating the volume of the structural crack as described above according to the embodiment of the present invention.
According to the technical scheme, the method for measuring and calculating the volume of the structural crack comprises the following steps: acquiring a plurality of crack sample images through a calibrated and registered depth camera and a monocular camera, and calculating corresponding crack volume values in the crack sample images; constructing a structural crack volume measuring and calculating model by taking the collected crack sample image as input and the calculated crack volume value as output; acquiring a crack RGB image of the structure to be detected through a monocular camera; and inputting the collected RGB image of the crack of the structure to be measured into the constructed structure crack volume measuring and calculating model for calculation to obtain the volume measuring and calculating information of the crack of the structure to be measured.
The method can greatly improve the detection precision and the detection efficiency, is suitable for measuring and calculating the volume of the cracks of various material structures, has strong universality, solves the problem that engineering cost personnel difficultly measure and calculate the volume of the cracks of the mass concrete structure, can realize real-time non-contact detection by applying monocular vision to the volume measurement and calculation of the cracks of the mass concrete structure, has low cost, high precision and easy operation, provides an effective way for damage repair, structure reinforcement and the like of the structure, and lays a foundation for realizing automation and intellectualization of intelligent maintenance of the structure. In addition, the invention also provides a corresponding realization device, equipment and a computer readable storage medium aiming at the method for measuring and calculating the structural crack volume based on monocular vision, so that the method has higher practicability, and the device, the equipment and the computer readable storage medium have corresponding advantages.
Drawings
In order to more clearly illustrate the embodiments of the present invention or technical solutions in related arts, the drawings used in the description of the embodiments or related arts will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a method for measuring and calculating a volume of a structural crack according to an embodiment of the present invention;
FIG. 2 is a diagram of the fast-RCNN network architecture of the structural crack volume estimation model according to the embodiment of the present invention;
fig. 3 is a specific flowchart of a method for measuring and calculating the volume of a structural crack according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a device for measuring and calculating the volume of a structural crack according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a method for measuring and calculating the volume of a structural crack, which comprises the following steps as shown in figure 1:
s101, collecting a plurality of crack sample images through a calibrated and registered depth camera and a monocular camera, and calculating corresponding crack volume values in the crack sample images; the crack sample image is obtained by registering an RGB (red, green and blue) image obtained by a monocular camera and a crack depth image obtained by a depth camera;
s102, constructing a structural crack volume measuring and calculating model by taking the collected crack sample image as input and the calculated crack volume value as output;
s103, acquiring a crack RGB image of the structure to be detected through a monocular camera; in practical application, any image acquisition equipment can be adopted to acquire a crack image of a structure to be detected, and then the acquired crack image is sent to a system; the image acquisition equipment can be a Digital Microscope (DM), the DM can quantitatively amplify and shoot times, effectively capture local characteristics of materials, accurately express the brightness and color range of the structure and output high-quality pictures;
and S104, inputting the collected RGB image of the crack of the structure to be measured into the constructed structure crack volume measurement model for calculation to obtain the volume measurement information of the crack of the structure to be measured. In practical application, the crack RGB image of the structure to be measured, which is acquired in real time, is transmitted to the trained structure crack volume measuring and calculating model through wired or wireless connection, and the result output by the structure crack volume measuring and calculating model is the volume information of the structure crack to be measured, so that the real-time measurement and calculation of the volume of the structure crack are realized.
The method for measuring and calculating the volume of the structural crack provided by the embodiment of the invention can greatly improve the detection precision and the detection efficiency, is suitable for measuring and calculating the volume of the structural crack of various materials, has strong universality, solves the problem that large-volume concrete structure crack volume is difficult to measure and calculate by engineering cost workers, can realize real-time non-contact detection by applying monocular vision to the volume measurement and calculation of the large-volume concrete structure crack, has low cost, high precision and easy operation, provides an effective way for damage repair, structural reinforcement and the like of the structure, and lays a foundation for realizing automation and intellectualization of intelligent maintenance of the structure.
It should be noted that before executing step S101, it is necessary to fix the positions of the depth camera and the monocular camera, where the image planes of the two are parallel to each other as much as possible, calibrate the depth camera and the monocular camera, and finally register the crack depth map obtained by the depth camera and the RGB map obtained by the monocular camera.
Specifically, the depth camera and the monocular camera may be calibrated by using a checkerboard method, and certainly, other methods may also be used for calibration, which are not limited herein.
When the chessboard method is adopted for Calibration, a Camera to be calibrated is used for shooting a plurality of chessboard pictures at different visual angles, and the inner reference of the Camera and the outer reference corresponding to each image are calculated by utilizing the Camera Calibration Toolbox of Matlab.
Then, registering the crack depth map and the RGB map, specifically including:
is provided with
To be the spatial coordinate of a point under the depth camera coordinates,
is the projected coordinate of the point on the image plane,
the internal reference matrix of the depth camera is known by a small hole imaging model and satisfies the following relations:
and is also provided with
Is the spatial coordinate of the same point under the monocular camera coordinates,
is the projection coordinate of the point on the RGB image plane,
is an internal reference matrix of the monocular camera. Due to the fact thatThe coordinates of the depth camera and the coordinates of the monocular camera are different and can be linked by a rotational-translational transformation, namely:
where R is the rotation matrix and T is the translation vector. Finally reused
To pair
Projecting to obtain corresponding RGB coordinates:
the external reference matrix is actually composed of a rotation matrix
（
) And translation vector
（
) The method is characterized in that a point P in a global coordinate system is transformed into a camera coordinate system, and a depth camera and a monocular camera are respectively transformed, and the following relations exist:
using P in formula (5) as
、
And
represented by formula (6) and can be obtained by:
comparing equation (3), the available rotation matrix and translation vector are:
in specific implementation, in the method for measuring and calculating the structural crack volume provided in the embodiment of the present invention, the step S101 may calculate a crack volume value corresponding to each crack sample image, and specifically includes the following steps:
firstly, acquiring the area of a crack cross section according to the number of pixels in a polygon formed by the crack cross section in an RGB image obtained by a monocular camera;
in practical application, the cross section of the crack obtained from the RGB diagram is an irregular arbitrary polygon, and the area of the cross section can be obtained by calculating the number of pixels in the polygon:
wherein,nthe number of pixels in the polygon;ηthe scaling factor between the horizontal direction and the actual size of the image can be calculated by dividing the actual physical size of the calibration block by the number of pixels of the calibration block in the image;
and then, according to a crack depth map obtained by the depth camera and the obtained cross section area of the crack, integrating to obtain a corresponding crack volume value in each crack sample image.
In specific implementation, the corresponding crack volume value in each crack sample image can be obtained by integrating the following formula:
wherein,
the volume value of the crack is taken as the value of the volume of the crack,
is as followsiThe area of the cross section of each crack; the crack depth map obtained by the depth camera is processed in the vertical directionhAre equally divided, each infinitesimal being
；
Is a scale factor of the crack depth map in the vertical direction to the actual size. It is understood that the cross-sectional shape of the crack has a one-to-one correspondence with the depth value of the crack.
In specific implementation, in the method for measuring and calculating the volume of the structural crack provided in the embodiment of the present invention, the step S102 of constructing a structural crack volume measurement model may specifically include: based on a transfer learning method, training a Faster-RCNN model by utilizing a training sample set to obtain a structural crack volume measurement model, wherein a network architecture is shown in FIG. 2; the training sample set at least comprises a plurality of crack sample images which are obtained by carrying out field actual measurement, model test or finite element calculation on the same material as the structural material to be tested. And calibrating each crack sample image with a corresponding crack depth map in advance, namely knowing crack volume information.
In practical application, components obtained through field actual measurement, model tests or finite element calculation are the same as the structure to be measured, crack sample images calibrated with corresponding crack depth maps in advance are input into a neural network model for training, and therefore the corresponding relation between the crack depth information and the picture characteristics of the structure to be measured is established. In addition, in practical application, the test component can also be made of any building structure material, the corresponding relation between the crack depth information of the component and the picture characteristics, which are actually measured on site, tested in a model or calculated in a finite element mode, is established, and the corresponding relation between the crack depth information of the structure to be tested and the picture characteristics is established by adopting deep learning migration after analysis and comparison. The crack depth information of the sample image can be obtained by any one of the related technologies such as field actual measurement, model test or finite element calculation. The more the crack depth information is, the more the crack sample images are, and the higher the calculation accuracy and precision of the trained structure crack volume measurement model is.
In practical application, a Faster-RCNN model can be designed in advance, and the weights of the fast-RCNN model which is designed in advance are migrated to a used neural network through adjustment and verification by adopting a migration learning method. Specifically, a training fast-RCNN model is trained by utilizing a training sample set and then transferred to a used neural network to obtain a structural crack volume measuring and calculating model.
It should be noted that, before training the fast-RCNN model by using the training sample set, the following steps may be included: each crack sample image in the training sample set is converted into a data set which is convenient for a deep learning format, specifically, the data set can be in a voc 2007 format, and the data set is used as an intensity level image feature of each crack sample image.
In specific implementation, in the method for measuring and calculating the volume of the structural crack provided in the embodiment of the present invention, in order to ensure the accuracy of the structural crack volume measurement model, before the step S104 is executed to input the acquired RGB image of the crack of the structure to be measured into the constructed structural crack volume measurement model for calculation, the method may further include: verifying the structural crack volume measurement model; as shown in fig. 3, the verifying the structure crack volume estimation model specifically includes the following steps:
s201, obtaining a model verification data set; the model verification data set comprises a plurality of verification sample images, and each verification sample image is calibrated with a corresponding crack depth map in advance and is a structural crack image which has known crack volume information and is the same as the structural material to be detected; verifying that the sample image and the crack sample image in the training sample set are images generated by the same method;
in practical application, the crack sample images in the training sample set can be divided into two parts, one part is used for training the neural network model, and the other part is used as the verification sample image;
s202, inputting each verification sample image into the constructed structural crack volume measurement model to obtain structural crack volume measurement information of each verification sample image;
in practical application, the crack volume measurement and calculation information is the crack volume measurement and calculation information of each verification sample image measured and calculated by the structural crack volume measurement and calculation model;
s203, calculating the accuracy of the structural crack volume measuring and calculating model according to the structural crack volume measuring and calculating information of each verification sample image, the known actual crack volume information and the total number of the verification sample images;
in practical application, for some application scenes with low requirements on precision, when the difference value between the calculated crack volume measurement and calculation information and the actual crack volume information is within an allowable deviation, the calculated crack volume measurement and calculation information and the actual crack volume information can be considered to be equivalent, namely, a default error does not exist; in a specific embodiment, therefore, the qualified number of the verification sample images with the difference value between the measured and calculated crack volume information and the known actual crack volume information being less than or equal to the preset deviation value can be counted, or the unqualified number of the verification sample images with the difference value between the measured and calculated crack volume information and the known actual crack volume information being greater than the preset deviation value can be counted, and then the qualified number is obtained; then calculating the ratio of the qualified number of the verification sample images to the total number of the verification sample images to be used as the accuracy of the structural crack volume measurement model;
s204, judging whether the accuracy of the structural crack volume measurement model is smaller than a preset threshold value or not;
if yes, go to step S205; if not, executing step S103;
in practical application, the preset deviation value and the preset threshold value can be set according to the requirement of the detection precision of a practical application scene, for example, in a high-precision detection scene, the preset threshold value is set to be 0, that is, if the difference value between the calculated crack volume measurement and calculation information and the actual crack volume information of one verified sample image is greater than the preset deviation value, the accuracy of the structural crack volume measurement and calculation model is determined not to pass, and the structural crack volume measurement and calculation model needs to be retrained;
s205, adding crack sample images in a training sample set;
in practical application, when the structural crack volume measuring and calculating model fails to be verified, a plurality of sample images can be added on the basis of a pre-training sample set, and the structural crack volume measuring and calculating model is retrained until the accuracy is not less than a preset threshold. The number of the increased sample images can be determined according to the accuracy of the structural crack volume measuring and calculating model and the required detection precision.
Based on the same inventive concept, the embodiment of the invention also provides a device for measuring and calculating the volume of the structural crack, and as the principle of solving the problem of the device is similar to that of the method for measuring and calculating the volume of the structural crack, the implementation of the device can refer to the implementation of the method for measuring and calculating the volume of the structural crack, and repeated details are not repeated.
In specific implementation, the device for measuring and calculating the volume of the structural crack provided by the embodiment of the present invention, as shown in fig. 4, specifically includes:
the sample data acquisition module 11 is used for acquiring a plurality of crack sample images through the calibrated and registered depth camera and the monocular camera, and calculating corresponding crack volume values in the crack sample images;
the model pre-construction module 12 is used for constructing a structural crack volume measurement model by taking the collected crack sample image as input and the calculated crack volume value as output; the model pre-construction module 12 may be specifically a module for obtaining a structural crack volume measurement model by training a neural network model with a training sample set based on a transfer learning method;
the to-be-detected image acquisition module 13 is used for acquiring a crack RGB image of the to-be-detected structure through a monocular camera;
and the crack volume measuring and calculating module 14 is used for inputting the acquired RGB image of the crack of the structure to be measured into the constructed structure crack volume measuring and calculating model for calculation to obtain the volume measuring and calculating information of the crack of the structure to be measured.
In the device for measuring and calculating the crack volume of the structure provided by the embodiment of the invention, the model can be quickly and effectively established through the interaction of the four modules, and the crack volume measuring and calculating information of the structure to be measured can be obtained.
In order to ensure the accuracy of the structural crack volume estimation model, the structural crack volume estimation device according to the embodiment of the present invention may further include: the model verification module 15 is used for verifying the structure crack volume measurement model; the model verification module 15 may specifically include:
the model verification data set acquisition sub-module is used for acquiring a model verification data set; the model verification data set comprises a plurality of verification sample images, and each verification sample image is calibrated with a corresponding crack depth map in advance and is a structural crack image which is the same as the structural material to be detected;
the crack volume measuring and calculating submodule is used for inputting each verification sample image into the constructed structural crack volume measuring and calculating model to obtain structural crack volume measuring and calculating information of each verification sample image;
the accuracy calculation submodule is used for calculating the accuracy of the structural crack volume measuring and calculating model according to the structural crack volume measuring and calculating information of each verification sample image, the known actual crack volume information and the total number of the verification sample images;
and the model retraining submodule is used for increasing the crack sample images in the training sample set when the accuracy of the structural crack volume measuring and calculating model is smaller than a preset threshold value, and retraining the structural crack volume measuring and calculating model until the accuracy is larger than or equal to the preset threshold value.
In practical application, the accuracy calculation sub-module may specifically count the qualified number of the verification sample images of which the difference between the calculated crack volume information and the known actual crack volume information is less than or equal to the preset deviation value, or count the unqualified number of the verification sample images of which the difference between the calculated crack volume information and the known actual crack volume information is greater than the preset deviation value, and then obtain the qualified number; and calculating the ratio of the qualified number to the total number to be used as the accuracy of the structural crack volume measurement model.
For more specific working processes of the modules, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Correspondingly, the embodiment of the invention also discloses a measuring and calculating device for the volume of the structural crack, which comprises a processor and a memory; the method for measuring and calculating the volume of the structural crack disclosed in the foregoing embodiments is implemented when the processor executes the computer program stored in the memory.
For more specific processes of the above method, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Further, the present invention also discloses a computer readable storage medium for storing a computer program; the computer program, when executed by a processor, implements the method for estimating structural crack volume as disclosed above.
For more specific processes of the above method, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device, the equipment and the storage medium disclosed by the embodiment correspond to the method disclosed by the embodiment, so that the description is relatively simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The method for measuring and calculating the volume of the structural crack provided by the embodiment of the invention comprises the following steps: acquiring a plurality of crack sample images through a calibrated and registered depth camera and a monocular camera, and calculating corresponding crack volume values in the crack sample images; constructing a structural crack volume measuring and calculating model by taking the collected crack sample image as input and the calculated crack volume value as output; acquiring a crack RGB image of the structure to be detected through a monocular camera; and inputting the collected RGB image of the crack of the structure to be measured into the constructed structure crack volume measuring and calculating model for calculation to obtain the volume measuring and calculating information of the crack of the structure to be measured. The method can greatly improve the detection precision and the detection efficiency, is suitable for measuring and calculating the crack volume of various material structures, has strong universality, solves the problem that large-volume concrete structure crack volume is difficult to measure and calculate by engineering cost personnel, can realize real-time non-contact detection, has low cost, high precision and easy operation, provides an effective way for damage repair, structure reinforcement and the like of the structure, and lays a foundation for realizing automation and intellectualization of intelligent maintenance of the structure. In addition, the invention also provides a corresponding realization device, equipment and a computer readable storage medium aiming at the method for measuring and calculating the structural crack volume based on monocular vision, so that the method has higher practicability, and the device, the equipment and the computer readable storage medium have corresponding advantages.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The method, the device, the equipment and the storage medium for measuring and calculating the volume of the structural crack provided by the invention are described in detail, a specific example is applied in the description to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (9)
1. A method for measuring and calculating the volume of a structural crack is characterized by comprising the following steps:
acquiring a plurality of crack sample images through a calibrated and registered depth camera and a monocular camera, and acquiring the area of the cross section of the crack according to the number of pixels in a polygon formed by the cross section of the crack in an RGB image obtained by the monocular camera;
according to a crack depth map obtained by a depth camera and the obtained cross section area of the crack, integrating to obtain a corresponding crack volume value in each crack sample image;
constructing a structural crack volume measuring and calculating model by taking the collected crack sample image as input and the calculated crack volume value as output;
acquiring a crack RGB image of the structure to be detected through a monocular camera;
and inputting the collected RGB image of the crack of the structure to be measured into the constructed crack volume measuring and calculating model of the structure to be measured for calculation, so as to obtain the volume measuring and calculating information of the crack of the structure to be measured.
2. The method for measuring and calculating the structural crack volume according to claim 1, wherein the corresponding crack volume value in each crack sample image is obtained by integrating the following formula:
wherein,
the volume value of the crack is taken as the value of the volume of the crack,
is as follows
The area of the cross section of each crack; performing crack depth map obtained by the depth camera in the vertical directionhAre equally divided, each infinitesimal being
；
Is a scale factor of the crack depth map in the vertical direction to the actual size.
3. The method for measuring and calculating the volume of the structural crack according to claim 2, wherein the construction of the structural crack volume measuring and calculating model specifically comprises:
training a Faster-RCNN model by utilizing a training sample set to obtain a structural crack volume measuring and calculating model; the training sample set at least comprises a plurality of crack sample images which are obtained by carrying out field actual measurement, model test or finite element calculation on the same material as the structural material to be tested.
4. The method for measuring and calculating the structural crack volume according to claim 3, wherein before inputting the collected RGB image of the crack of the structure to be measured into the constructed structural crack volume measuring and calculating model for calculation, the method further comprises:
obtaining a model verification dataset; the model verification data set comprises a plurality of verification sample images, and each verification sample image is calibrated with a corresponding crack depth map in advance and is a structural crack image which is the same as the structural material to be detected;
inputting each verification sample image into the constructed structural crack volume measurement model to obtain structural crack volume measurement information of each verification sample image;
calculating the accuracy of the structural crack volume calculation model according to the structural crack volume calculation information, the known actual crack volume information and the total number of the verification sample images of each verification sample image;
judging whether the accuracy of the structural crack volume measuring and calculating model is smaller than a preset threshold value or not;
if so, increasing the crack sample images in the training sample set, and retraining the structural crack volume measuring and calculating model until the accuracy is greater than or equal to the preset threshold; and if not, using the structural crack volume measuring and calculating model for subsequently measuring and calculating the volume information of the structural crack to be measured.
5. The method for measuring and calculating the structural crack volume according to claim 4, wherein the accuracy of the structural crack volume measurement model is calculated according to the structural crack volume measurement information, the known actual crack volume information and the total number of the verification sample images of each verification sample image, and specifically comprises:
counting the qualified number of the verification sample images of which the difference value between the structure crack measurement information and the known actual crack volume information of each verification sample image is less than or equal to a preset deviation value;
and calculating the ratio of the qualified number and the total number of the verification sample images to be used as the accuracy of the structural crack volume measurement model.
6. A device for measuring and calculating the volume of a structural crack is characterized by comprising:
the sample data acquisition module is used for acquiring a plurality of crack sample images through the calibrated and registered depth camera and the monocular camera, acquiring the area of the cross section of the crack according to the number of pixels in a polygon formed by the cross section of the crack in the RGB image obtained by the monocular camera, and integrating to obtain the corresponding crack volume value in each crack sample image according to the crack depth image obtained by the depth camera and the acquired area of the cross section of the crack;
the model pre-construction module is used for constructing a structural crack volume measurement model by taking the collected crack sample image as input and the calculated crack volume value as output;
the to-be-detected image acquisition module is used for acquiring a crack RGB image of the to-be-detected structure through a monocular camera;
and the crack volume measuring and calculating module is used for inputting the acquired RGB image of the crack of the structure to be measured into the constructed crack volume measuring and calculating model of the structure to be measured for calculation so as to obtain the volume measuring and calculating information of the crack of the structure to be measured.
7. The device for measuring and calculating the volume of structural cracks according to claim 6, further comprising: the model verification module is used for verifying the structural crack volume measurement and calculation model; the model verification module includes:
the model verification data set acquisition sub-module is used for acquiring a model verification data set; the model verification data set comprises a plurality of verification sample images, and each verification sample image is calibrated with a corresponding crack depth map in advance and is a structural crack image which is the same as the structural material to be detected;
the crack volume measuring and calculating submodule is used for inputting each verification sample image into the constructed structural crack volume measuring and calculating model to obtain structural crack volume measuring and calculating information of each verification sample image;
the accuracy calculation submodule is used for calculating the accuracy of the structural crack volume measuring and calculating model according to the structural crack volume measuring and calculating information of each verified sample image, the known actual crack volume information and the total number of the verified sample images;
and the model retraining submodule is used for increasing the crack sample images in the training sample set when the accuracy of the structural crack volume measuring and calculating model is smaller than a preset threshold value, and retraining the structural crack volume measuring and calculating model until the accuracy is larger than or equal to the preset threshold value.
8. An apparatus for measuring and calculating structural crack volume, comprising a processor and a memory, wherein the processor implements the method for measuring and calculating structural crack volume according to any one of claims 1 to 5 when executing a computer program stored in the memory.
9. A computer-readable storage medium for storing a computer program, wherein the computer program, when being executed by a processor, implements the method for estimating a structural crack volume according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011033662.0A CN111932642B (en) | 2020-09-27 | 2020-09-27 | Method, device and equipment for measuring and calculating volume of structural crack and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011033662.0A CN111932642B (en) | 2020-09-27 | 2020-09-27 | Method, device and equipment for measuring and calculating volume of structural crack and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111932642A CN111932642A (en) | 2020-11-13 |
| CN111932642B true CN111932642B (en) | 2021-01-01 |
Family
ID=73334234
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011033662.0A Active CN111932642B (en) | 2020-09-27 | 2020-09-27 | Method, device and equipment for measuring and calculating volume of structural crack and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111932642B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115077832B (en) * | 2022-07-28 | 2022-11-08 | 西安交通大学 | Method for measuring vibration fatigue damage of three-dimensional surface of high-temperature-resistant component of airplane |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10753881B2 (en) * | 2016-05-27 | 2020-08-25 | Purdue Research Foundation | Methods and systems for crack detection |
| CN108535267A (en) * | 2018-04-13 | 2018-09-14 | 江苏南通二建集团有限公司 | A kind of Volume Concrete Cracks monitoring device |
| CN110910343A (en) * | 2019-09-29 | 2020-03-24 | 北京建筑大学 | Method and device for detecting pavement cracks and computer equipment |
| CN110853138B (en) * | 2019-11-21 | 2023-08-18 | 科吉思石油技术咨询(北京)有限公司 | Construction method of dual-medium carbonate pore-crack dual-network model |
| CN111259845A (en) * | 2020-01-21 | 2020-06-09 | 河海大学 | Concrete structure surface crack identification method |
-
2020
- 2020-09-27 CN CN202011033662.0A patent/CN111932642B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111932642A (en) | 2020-11-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113252700B (en) | Structural crack detection method, equipment and system | |
| CN108960135B (en) | Dense ship target accurate detection method based on high-resolution remote sensing image | |
| CN111189403B (en) | Tunnel deformation monitoring method and device and computer readable storage medium | |
| CN107274453A (en) | Video camera three-dimensional measuring apparatus, system and method for a kind of combination demarcation with correction | |
| CN109858551A (en) | Method, apparatus, equipment and medium based on image recognition detection structure stress | |
| CN113012047B (en) | Dynamic camera coordinate mapping establishing method and device and readable storage medium | |
| CN113470562B (en) | OLED screen sub-pixel brightness extraction method based on imaging brightness meter | |
| CN109035292A (en) | Moving target detecting method and device based on deep learning | |
| CN116597013B (en) | Satellite image geometric calibration method based on different longitude and latitude areas | |
| CN113569647B (en) | AIS-based ship high-precision coordinate mapping method | |
| CN109214254B (en) | Method and device for determining displacement of robot | |
| CN111932642B (en) | Method, device and equipment for measuring and calculating volume of structural crack and storage medium | |
| CN112308926A (en) | Camera external reference calibration method without public view field | |
| CN112149356A (en) | Method, device, equipment and medium for predicting structural crack propagation path | |
| CN101894369B (en) | Real-time method for calculating camera focal length from image sequence | |
| Lascelles et al. | Automated digital photogrammetry: a valuable tool for small‐scale geomorphological research for the non‐photogrammetrist? | |
| CN110533713A (en) | Bridge Crack width high-precision measuring method and measuring device | |
| CN114595238A (en) | Vector-based map processing method and device | |
| CN109829951B (en) | Parallel equipotential detection method and device and automatic driving system | |
| CN111207683A (en) | Tunnel deformation monitoring method and device and computer readable storage medium | |
| CN114187179B (en) | Remote sensing image simulation generation method and system based on video monitoring | |
| CN116152325A (en) | Road traffic high slope stability monitoring method based on monocular video | |
| CN116597132A (en) | Non-contact measuring method and system for static and dynamic displacement of steel structure node | |
| CN112053331B (en) | Bridge crack detection method based on image superposition and crack information fusion | |
| Kuo et al. | Infrastructure Inspection Using an Unmanned Aerial System (UAS) With Metamodeling-Based Image Correction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |