Intelligent Assessment of Pavement Structural Conditions: A Novel FeMViT Classification Network for GPR Images
Pages 13511 - 13523
Abstract
Traditional road structural detection and evaluation is inefficient, imprecise, and destructive. To address these issues, a feature-enhanced multiscale vision transformer (FeMViT) for road distress classification from ground penetrating radar (GPR) images was proposed. FeMViT model used the feature-enhanced feature pyramid network (FPN) and feature enrichment module (FEM) to extract the distress better features on GPR images. The pooling attention was also modified using the residual pooling connection to reduce computational complexity and memory usage. Experimental results showed that this model further realized the comprehensive improvement of classification indexes for road distresses. The accuracy and <inline-formula> <tex-math notation="LaTeX">$\boldsymbol { F}_{1}$ </tex-math></inline-formula> score of the overall classification result was 91.9% and 90.8%, improved by 10.4% and 7.1% compared to the original Transformer, respectively. Misattribution and visualization analysis provided ideas for improvement directions. The internal distress rate (<inline-formula> <tex-math notation="LaTeX">$\boldsymbol {IDR}$ </tex-math></inline-formula>) and internal pavement structural integrity score (<inline-formula> <tex-math notation="LaTeX">$\boldsymbol {IPSI}$ </tex-math></inline-formula>) indexes of structural integrity were determined based on GPR images. Field tests suggested a good correlation between the structural strength and integrity indexes of asphalt pavement. This illustrates that the proposed method is reliable and could provide a more comprehensive approach to the structural condition assessment of asphalt pavement.
References
[1]
B. Cui, H. Wang, X. Gu, and D. Hu, “Study of the inter-diffusion characteristics and cracking resistance of virgin-aged asphalt binders using molecular dynamics simulation,” Construct. Building Mater., vol. 351, Oct. 2022, Art. no.
[2]
Z. Liu, W. Wu, X. Gu, S. Li, L. Wang, and T. Zhang, “Application of combining YOLO models and 3D GPR images in road detection and maintenance,” Remote Sens., vol. 13, no. 6, p. 1081, Mar. 2021.
[3]
C. Xiong, J. Yu, and X. Zhang, “Use of NDT systems to investigate pavement reconstruction needs and improve maintenance treatment decision-making,” Int. J. Pavement Eng., vol. 24, no. 1, pp. 1–11, Dec. 10, 2021.
[4]
M. Solla, V. Pérez-Gracia, and S. Fontul, “A review of GPR application on transport infrastructures: Troubleshooting and best practices,” Remote Sens., vol. 13, no. 4, p. 672, Feb. 2021.
[5]
Z. Liu, X. Gu, H. Ren, S. Li, and Q. Dong, “Permanent deformation monitoring and remaining life prediction of asphalt pavement combining full-scale accelerated pavement testing and FEM,” Struct. Control Health Monitor., vol. 2023, pp. 1–19, Jun. 2023.
[6]
Z. Fan et al., “Automatic crack detection on road pavements using encoder–decoder architecture,” Materials, vol. 13, no. 13, p. 2960, Jul. 2020.
[7]
C. Li et al., “CrackCLF: Automatic pavement crack detection based on closed-loop feedback,” IEEE Trans. Intell. Transp. Syst., pp. 1–16, Nov. 2023.
[8]
Z. Liu, X. Gu, and H. Ren, “Rutting prediction of asphalt pavement with semi-rigid base: Numerical modeling on laboratory to accelerated pavement testing,” Construct. Building Mater., vol. 375, Apr. 2023, Art. no.
[9]
Z. Liu, X. Gu, Q. Dong, S. Tu, and S. Li, “3D visualization of airport pavement quality based on BIM and webGL integration,” J. Transp. Eng., Part B, Pavements, vol. 147, no. 3, pp. 1–14, Sep. 2021.
[10]
Z. Liu, X. Y. Gu, and Q. Dong, “Multi-scale 3D display of the internal quality of the pavement based on BIM,” in Proc. CICTP, 2019, pp. 4265–4273.
[11]
B. Cui and H. Wang, “Cross-scale analysis of asphalt binder tensile fracture using molecular dynamics simulation,” Construct. Building Mater., vol. 426, May 2024, Art. no.
[12]
Z. Liu and X. Gu, “Performance evaluation of full-scale accelerated pavement using NDT and laboratory tests: A case study in Jiangsu, China,” Case Stud. Construct. Mater., vol. 18, Jul. 2023, Art. no.
[13]
Z. Liu, B. Cui, Q. Yang, and X. Gu, “Sensor-based structural health monitoring of asphalt pavements with semi-rigid bases combining accelerated pavement testing and a falling weight deflectometer test,” Sensors, vol. 24, no. 3, p. 994, Feb. 2024.
[14]
S. Wang and I. L. Al-Qadi, “Impact and removal of ground-penetrating radar vibration on continuous asphalt concrete pavement density prediction,” IEEE Trans. Geosci. Remote Sens., vol. 60, Aug. 2021, Art. no.
[15]
Z. Liu, Q. Yang, and X. Gu, “Assessment of pavement structural conditions and remaining life combining accelerated pavement testing and ground-penetrating radar,” Remote Sens., vol. 15, no. 18, p. 4620, Sep. 2023.
[16]
J. Long, Q. Luo, Z. Liu, and Z. Zhu, “Road distress detection and maintenance evaluation based on ground penetrating radar,” in Advances in Civil Function Structure and Industrial Architecture. Boca Raton, FL, USA: CRC Press, 2022, pp. 474–481.
[17]
L. Wang, X. Gu, Z. Liu, W. Wu, and D. Wang, “Automatic detection of asphalt pavement thickness: A method combining GPR images and improved Canny algorithm,” Measurement, vol. 196, Jun. 2022, Art. no.
[18]
Z. Liu, X. Gu, Y. Chen, and Y. Chen, “System architecture and key technologies for the whole life cycle of smart road,” J. Phys. Conf. Ser., vol. 1972, no. 1, Jul. 2021, Art. no.
[19]
M.-S. Kang and Y.-K. An, “Frequency–Wavenumber analysis of deep learning-based super resolution 3D GPR images,” Remote Sens., vol. 12, no. 18, p. 3056, Sep. 2020.
[20]
S. Li et al., “Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm,” Construct. Building Mater., vol. 273, Mar. 2021, Art. no.
[21]
T. Yamaguchi, T. Mizutani, K. Meguro, and T. Hirano, “Detecting subsurface voids from GPR images by 3-D convolutional neural network using 2-D finite difference time domain method,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 15, pp. 3061–3073, 2022.
[22]
S. Zhao and I. L. Al-Qadi, “Super-resolution of 3-D GPR signals to estimate thin asphalt overlay thickness using the XCMP method,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 2, pp. 893–901, Feb. 2019.
[23]
X. Liang, X. Yu, C. Chen, Y. Jin, and J. Huang, “Automatic classification of pavement distress using 3D ground-penetrating radar and deep convolutional neural network,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 11, pp. 22269–22277, Nov. 2022.
[24]
T. Zhang, D. Wang, A. Mullins, and Y. Lu, “Integrated APC-GAN and AttuNet framework for automated pavement crack pixel-level segmentation: A new solution to small training datasets,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 4, pp. 4474–4481, Apr. 2023.
[25]
J. Huang et al., “A deep learning framework based on improved self-supervised learning for ground-penetrating radar tunnel lining inspection,” Comput.-Aided Civil Infrastruct. Eng., vol. 39, no. 6, pp. 814–833, Mar. 2024.
[26]
G. Zhu et al., “A lightweight encoder–decoder network for automatic pavement crack detection,” Comput.-Aided Civil Infrastruct. Eng., early access, Oct. 2023.
[27]
Y. Chen, X. Gu, Z. Liu, and J. Liang, “A fast inference vision transformer for automatic pavement image classification and its visual interpretation method,” Remote Sens., vol. 14, no. 8, p. 1877, Apr. 2022.
[28]
Z. Tong, D. Yuan, J. Gao, Y. Wei, and H. Dou, “Pavement-distress detection using ground-penetrating radar and network in networks,” Construct. Building Mater., vol. 233, Feb.10, 2020, Art. no.
[29]
Z. Tong, D. Yuan, J. Gao, and Z. Wang, “Pavement defect detection with fully convolutional network and an uncertainty framework,” Comput.-Aided Civil Infrastruct. Eng., vol. 35, no. 8, pp. 832–849, Aug. 2020.
[30]
Z. Tong, J. Gao, and H. Zhang, “Innovative method for recognizing subgrade defects based on a convolutional neural network,” Construct. Building Mater., vol. 169, pp. 69–82, Apr.30, 2018.
[31]
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 779–788.
[32]
T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 2999–3007.
[33]
K. Duan, S. Bai, L. Xie, H. Qi, Q. Huang, and Q. Tian, “CenterNet: Keypoint triplets for object detection,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2019, pp. 6568–6577.
[34]
D. Wang, Z. Liu, X. Gu, W. Wu, Y. Chen, and L. Wang, “Automatic detection of pothole distress in asphalt pavement using improved convolutional neural networks,” Remote Sens., vol. 14, no. 16, p. 3892, Aug. 2022.
[35]
D. Wang, Z. Liu, X. Gu, and W. Wu, “Feature extraction and segmentation of pavement distress using an improved hybrid task cascade network,” Int. J. Pavement Eng., vol. 24, no. 1, Dec. 2023, Art. no.
[36]
Z. Liu, X. Gu, J. Chen, D. Wang, Y. Chen, and L. Wang, “Automatic recognition of pavement cracks from combined GPR B-scan and C-scan images using multiscale feature fusion deep neural networks,” Autom. Construct., vol. 146, Feb. 2023, Art. no.
[37]
Z. Liu, X. Gu, H. Yang, L. Wang, Y. Chen, and D. Wang, “Novel YOLOV3 model with structure and hyperparameter optimization for detection of pavement concealed cracks in GPR images,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 11, pp. 22258–22268, Nov. 2022.
[38]
Z. Qiu, Z. Zhao, S. Chen, J. Zeng, Y. Huang, and B. Xiang, “Application of an improved YOLOV5 algorithm in real-time detection of foreign objects by ground penetrating radar,” Remote Sens., vol. 14, no. 8, p. 1895, Apr. 2022.
[39]
Z. Fan et al., “Ensemble of deep convolutional neural networks for automatic pavement crack detection and measurement,” Coatings, vol. 10, no. 2, p. 152, Feb. 2020.
[40]
T. Zhang, D. Wang, and Y. Lu, “ECSNet: An accelerated real-time image segmentation CNN architecture for pavement crack detection,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 12, pp. 15105–15112, Dec. 2023.
[41]
A. A. Zhang et al., “Intelligent pixel-level detection of multiple distresses and surface design features on asphalt pavements,” Computer-Aided Civil Infrastructure Eng., vol. 37, no. 13, pp. 1654–1673, Nov. 2022.
[42]
P. J. Chun, M. Suzuki, and Y. Kato, “Iterative application of generative adversarial networks for improved buried pipe detection from images obtained by ground-penetrating radar,” Comput.-Aided Civil Infrastruct. Eng., vol. 38, no. 17, pp. 2472–2490, Nov. 2023.
[43]
S. Li, X. Cui, L. Guo, L. Zhang, X. Chen, and X. Cao, “Enhanced automatic root recognition and localization in GPR images through a YOLOV4-based deep learning approach,” IEEE Trans. Geosci. Remote Sens., vol. 60, Jun. 2022, Art. no.
[44]
Y. Yue et al., “Automatic recognition of defects behind railway tunnel linings in GPR images using transfer learning,” Measurement, vol. 224, Jan. 2024, Art. no.
[45]
H. Qin, D. Zhang, Y. Tang, and Y. Wang, “Automatic recognition of tunnel lining elements from GPR images using deep convolutional networks with data augmentation,” Autom. Construct., vol. 130, Oct. 2021, Art. no.
[46]
F. Hou, W. Lei, S. Li, J. Xi, M. Xu, and J. Luo, “Improved mask R-CNN with distance guided intersection over union for GPR signature detection and segmentation,” Autom. Construct., vol. 121, Jan. 2021, Art. no.
[47]
Z. Liu et al., “Automatic pixel-level detection of vertical cracks in asphalt pavement based on GPR investigation and improved mask R-CNN,” Autom. Construct., vol. 146, Feb. 2023, Art. no.
[48]
R. Fan, F. Li, W. Han, J. Yan, J. Li, and L. Wang, “Fine-scale urban informal settlements mapping by fusing remote sensing images and building data via a transformer-based multimodal fusion network,” IEEE Trans. Geosci. Remote Sens., vol. 60, Sep. 2022, Art. no.
[49]
K. Han et al., “A survey on vision transformer,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 1, pp. 87–110, Jan. 2023.
[50]
Z. Liu, X. Gu, W. Wu, X. Zou, Q. Dong, and L. Wang, “GPR-based detection of internal cracks in asphalt pavement: A combination method of DeepAugment data and object detection,” Measurement, vol. 197, Jun.30, 2022, Art. no.
[51]
G. Jin, G. Zang, W. Cai, H. Lu, and J. Zhao, “A quantitative evaluation method for pavement structural integrity based on ground penetrating radar,” Highway, vol. 65, no. 5, pp. 16–20, 2020.
[52]
Q. Zeng, L. Chen, P. Gao, G. Jin, G. Zang, and X. Sun, “Pavement structure performance evaluation method based on GPR and TSD,” Highway, vol. 67, no. 12, pp. 107–112, 2022.
[53]
J. Bai et al., “Hyperspectral image classification based on multibranch attention transformer networks,” IEEE Trans. Geosci. Remote Sens., vol. 60, Aug. 2022, Art. no.
[54]
H. Fan et al., “Multiscale vision transformers,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., May 2021, pp. 6804–6815.
[55]
A. Khan et al., “A survey of the vision transformers and their CNN-transformer based variants,” Artif. Intell. Rev., vol. 56, no. 3, pp. 2917–2970, Dec. 2023.
[56]
Z. Liu, Q. Yang, A. Wang, and X. Gu, “Vehicle driving safety of underground interchanges using a driving simulator and data mining analysis,” Infrastructures, vol. 9, no. 2, p. 28, Feb. 2024.
[57]
Q. Liu, B. Cui, and Z. Liu, “Air quality class prediction using machine learning methods based on monitoring data and secondary modeling,” Atmosphere, vol. 15, no. 5, p. 553, Apr. 2024.
[58]
Z. Liu, Y. Chen, X. Gu, J. K. W. Yeoh, and Q. Zhang, “Visibility classification and influencing-factors analysis of airport: A deep learning approach,” Atmos. Environ., vol. 278, Jun. 2022, Art. no.
Index Terms
- Intelligent Assessment of Pavement Structural Conditions: A Novel FeMViT Classification Network for GPR Images
Index terms have been assigned to the content through auto-classification.
Recommendations
Application of Freeman decomposition to full polarimetric GPR for improving subsurface target classification
Migration technique can reconstruct the subsurface target imaging from record ground penetrating radar (GPR) data. Depending on the reconstructed imaging, we possibly distinguish subsurface target by the geometrical feature. But some different targets ...
Finite element model for new continuous reinforced concrete pavement (CRCP) using GFRP bars
MS'06: Proceedings of the 17th IASTED international conference on Modelling and simulationSubstitution of conventional steel reinforcing bars with Glass Fibre Reinforced Polymer (GFRP) bars in Continuously Reinforced Concrete Pavement (CRCP) gives solutions to the problems caused by corrosion of reinforcement. Concrete volume change is known ...
Research on the Service Performance Index System of Asphalt Pavement Based on Intelligent Sensing
IoTAAI '23: Proceedings of the 2023 5th International Conference on Internet of Things, Automation and Artificial IntelligenceThis study comprehensively evaluates the service performance of asphalt pavement and establishes an intelligent sensing based asphalt pavement service performance indicator system. The principles for establishing a monitoring index system for asphalt ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
1524-9050 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 01 October 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 27 Jan 2025