A Fast and Robust Lane Detection via Online Re-Parameterization and Hybrid Attention
<p>Overall structure of the lane detection model.</p> "> Figure 2
<p>Online re-parameterization conversion process.</p> "> Figure 3
<p>Online re-parameterization convolution module.</p> "> Figure 4
<p>Detailed structure of the Efficient Channel Attention Module.</p> "> Figure 5
<p>Detailed structure of the Position Attention Module.</p> "> Figure 6
<p>Row anchor classification diagram.</p> "> Figure 7
<p>The detection results for the three models are presented. The first row is straight road scenes, the second row is distant curved road scenes, the third row is near-field occlusion scenes, and the fourth row is multiple occlusion scenes.</p> "> Figure 8
<p>Lane detection visualization results across nine distinct traffic scenarios.</p> ">
Abstract
:1. Introduction
- We propose a lane detection model that integrates online re-parameterized ResNet and row-anchor classification. This model possesses efficient inference speed, ensuring real-time detection under various complex traffic scenarios.
- A hybrid attention module combining position and channel attention is designed, which captures feature information more comprehensively, enabling the model to focus on the slender lane line details in the image.
- Comparative experiments are performed on the TuSimple and CULane datasets with other lane detection models. Our model achieves better detection results. The experiments demonstrate that the proposed model meets the accuracy and robustness requirements for lane detection.
2. Related Work
2.1. Lane Detection Based on Deep Learning
2.2. Re-Parameterization
2.3. Attention Mechanisms
3. Proposed Method
3.1. Online Re-Parameterization
3.2. Hybrid Attention Module
3.3. Row Anchor Classification
3.4. Loss Function
4. Experiment
4.1. Datasets
4.2. Experimental Environment
4.3. Evaluation Indicators
4.4. Module Comparison Experiment
4.5. Ablation Experiment
4.6. Performance Comparison of Different Models
4.7. Robustness Testing
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Lamssaggad, A.; Benamar, N.; Hafid, A.S.; Msahli, M. A survey on the current security landscape of intelligent transportation systems. IEEE Access 2021, 9, 9180–9208. [Google Scholar] [CrossRef]
- Kumar, S.; Jailia, M.; Varshney, S. A Comparative Study of Deep Learning based Lane Detection Methods. In Proceedings of the 2022 9th International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India, 23–25 March 2022; pp. 579–584. [Google Scholar]
- He, Y.; Wang, H.; Zhang, B. Color-based road detection in urban traffic scenes. IEEE Trans. Intell. Transp. Syst. 2004, 5, 309–318. [Google Scholar]
- Chiu, K.; Lin, S. Lane detection using color-based segmentation. In Proceedings of the IEEE Intelligent Vehicles Symposium, Las Vegas, NV, USA, 6–8 June; 2005; pp. 706–711. [Google Scholar]
- Tapia-Espinoza, R.; Torres-Torriti, M. A comparison of gradient versus color and texture analysis for lane detection and tracking. In Proceedings of the 2009 6th Latin American Robotics Symposium (LARS 2009), Valparaiso, Chile, 29–30 October 2009; pp. 1–6. [Google Scholar]
- Li, Z.; Ma, H.; Liu, Z. Road lane detection with gabor filters. In Proceedings of the 2016 International Conference on Information System and Artificial Intelligence (ISAI), Hong Kong, China, 24–26 June 2016; pp. 436–440. [Google Scholar]
- Gao, W.; Zhang, X.; Yang, L.; Liu, H. An improved Sobel edge detection. In Proceedings of the 2010 3rd International Conference on Computer Science and Information Technology, Chengdu, China, 9–11 July 2010; pp. 67–71. [Google Scholar]
- Xuan, L.; Hong, Z. An improved canny edge detection algorithm. In Proceedings of the 2017 8th IEEE International Conference on Software Engineering and Service Science (ICSESS), Beijing, China, 24–26 November 2017; pp. 275–278. [Google Scholar]
- Luo, S.; Zhang, X.; Hu, J.; Xu, J. Multiple lane detection via combining complementary structural constraints. IEEE Trans. Intell. Transp. Syst. 2020, 22, 7597–7606. [Google Scholar] [CrossRef]
- Bisht, S.; Sukumar, N.; Sumathi, P. Integration of Hough Transform and Inter-Frame Clustering for Road Lane Detection and Tracking. In Proceedings of the 2022 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Ottawa, ON, Canada, 16–19 May 2022; pp. 1–6. [Google Scholar]
- Kim, J.; Lee, M. Robust lane detection based on convolutional neural network and random sample consensus. In Neural Information Processing, Proceedings of the 21st International Conference, ICONIP 2014, Kuching, Malaysia, 3–6 November 2014; Proceedings, Part I 21; Springer: Berlin/Heidelberg, Germany, 2014; pp. 454–461. [Google Scholar]
- Sukumar, N.; Sumathi, P. A Robust Vision-based Lane Detection using RANSAC Algorithm. In Proceedings of the 2022 IEEE Global Conference on Computing, Power and Communication Technologies (GlobConPT), New Delhi, India, 23–25 September 2022; pp. 1–5. [Google Scholar]
- Cai, H.; Hu, Z.; Huang, G.; Zhu, D. Robust road lane detection from shape and color feature fusion for vehicle self-localization. In Proceedings of the 2017 4th International Conference on Transportation Information and Safety (ICTIS), Banff, AB, Canada, 8–10 August 2017; pp. 1009–1014. [Google Scholar]
- Guo, J.; Wei, Z.; Miao, D. Lane detection method based on improved RANSAC algorithm. In Proceedings of the 2015 IEEE Twelfth International Symposium on Autonomous Decentralized Systems, Taichung, Taiwan, 25–27 March 2015; pp. 285–288. [Google Scholar]
- Lecun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
- Lee, S.; Kim, J.; Shin Yoon, J.; Shin, S.; Bailo, O.; Kim, N.; Lee, T.; Seok Hong, H.; Han, S.; So Kweon, I. Vpgnet: Vanishing point guided network for lane and road marking detection and recognition. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 1947–1955. [Google Scholar]
- Neven, D.; De Brabandere, B.; Georgoulis, S.; Proesmans, M.; Van Gool, L. Towards end-to-end lane detection: An instance segmentation approach. In Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018; pp. 286–291. [Google Scholar]
- Pan, X.; Shi, J.; Luo, P.; Wang, X.; Tang, X. Spatial as deep: Spatial cnn for traffic scene understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, 2–7 February 2018. [Google Scholar]
- Hou, Y.; Ma, Z.; Liu, C.; Loy, C.C. Learning lightweight lane detection cnns by self attention distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 1013–1021. [Google Scholar]
- Xu, H.; Wang, S.; Cai, X.; Zhang, W.; Liang, X.; Li, Z. Curvelane-nas: Unifying lane-sensitive architecture search and adaptive point blending. In Computer Vision-ECCV 2020, Proceedings of the 16th European Conference, Glasgow, UK, 23–28 August 2020; Proceedings, Part XV 16; Springer: Berlin/Heidelberg, Germany, 2020; pp. 689–704. [Google Scholar]
- Ko, Y.; Lee, Y.; Azam, S.; Munir, F.; Jeon, M.; Pedrycz, W. Key points estimation and point instance segmentation approach for lane detection. IEEE Trans. Intell. Transp. Syst. 2021, 23, 8949–8958. [Google Scholar] [CrossRef]
- Tabelini, L.; Berriel, R.; Paixao, T.M.; Badue, C.; De Souza, A.F.; Oliveira-Santos, T. Polylanenet: Lane estimation via deep polynomial regression. In Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, 10–15 January 2021; pp. 6150–6156. [Google Scholar]
- Feng, Z.; Guo, S.; Tan, X.; Xu, K.; Wang, M.; Ma, L. Rethinking efficient lane detection via curve modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 21–24 June 2022; pp. 17062–17070. [Google Scholar]
- Wang, J.; Ma, Y.; Huang, S.; Hui, T.; Wang, F.; Qian, C.; Zhang, T. A keypoint-based global association network for lane detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 21–24 June 2022; pp. 1392–1401. [Google Scholar]
- Tabelini, L.; Berriel, R.; Paixao, T.M.; Badue, C.; De Souza, A.F.; Oliveira-Santos, T. Keep your eyes on the lane: Real-time attention-guided lane detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual, 19–25 June 2021; pp. 294–302. [Google Scholar]
- Liu, L.; Chen, X.; Zhu, S.; Tan, P. Condlanenet: A top-to-down lane detection framework based on conditional convolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual, 11–17 October 2021; pp. 3773–3782. [Google Scholar]
- Qin, Z.; Zhang, P.; Li, X. Ultra fast structure-aware deep lane detection. In Proceedings of the Computer Vision—ECCV 2020: 16th European Conference, Glasgow, UK, 23–28 August 2020; pp. 276–291. [Google Scholar]
- Zheng, T.; Huang, Y.; Liu, Y.; Tang, W.; Yang, Z.; Cai, D.; He, X. Clrnet: Cross layer refinement network for lane detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 21–24 June 2022; pp. 898–907. [Google Scholar]
- Hu, M.; Feng, J.; Hua, J.; Lai, B.; Huang, J.; Gong, X.; Hua, X. Online convolutional re-parameterization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 21–24 June 2022; pp. 568–577. [Google Scholar]
- Ding, X.; Guo, Y.; Ding, G.; Han, J. Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 1911–1920. [Google Scholar]
- Ding, X.; Zhang, X.; Ma, N.; Han, J.; Ding, G.; Sun, J. Repvgg: Making vgg-style convnets great again. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual, 19–25 June 2021; pp. 13733–13742. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Ding, X.; Zhang, X.; Han, J.; Ding, G. Diverse branch block: Building a convolution as an inception-like unit. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual, 19–25 June 2021; pp. 10886–10895. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 22 June 2018; pp. 7132–7141. [Google Scholar]
- Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; Hu, Q. ECA-Net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–18 June 2020; pp. 11534–11542. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.; Kweon, I.S. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Fu, J.; Liu, J.; Tian, H.; Li, Y.; Bao, Y.; Fang, Z.; Lu, H. Dual attention network for scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 16–19 June 2019; pp. 3146–3154. [Google Scholar]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv 2017, arXiv:1704.04861. [Google Scholar]
- Qin, Z.; Zhang, P.; Wu, F.; Li, X. Fcanet: Frequency channel attention networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual, 11–17 October 2021; pp. 783–792. [Google Scholar]
- Ren, F.; Zhou, H.; Yang, L.; Liu, F.; He, X. ADPNet: Attention based dual path network for lane detection. J. Vis. Commun. Image Represent. 2022, 87, 103574. [Google Scholar] [CrossRef]
- Song, Y.; Huang, T.; Fu, X.; Jiang, Y.; Xu, J.; Zhao, J.; Yan, W.; Wang, X. A Novel Lane Line Detection Algorithm for Driverless Geographic Information Perception Using Mixed-Attention Mechanism ResNet and Row Anchor Classification. ISPRS Int. J. Geo-Inf. 2023, 12, 132. [Google Scholar] [CrossRef]
Dataset | Frame | Train | Validation | Test | Resolution |
---|---|---|---|---|---|
TuSimple | 6408 | 3268 | 358 | 2782 | 1280 × 720 |
CULane | 133,235 | 88,880 | 9675 | 34,680 | 1640 × 590 |
Model | FLOPs/G | Params/M | Params Size/MB |
---|---|---|---|
Resnet18 | 9.389 | 96.369 | 367.62 |
Resnet_OREPA | 0.235 | 85.375 | 325.68 |
Module | ACC | FP | FN | F1 |
---|---|---|---|---|
PAM [37] | 95.81 | 2.77 | 4.57 | 96.31 |
ECANet [35] | 95.87 | 2.81 | 4.61 | 96.27 |
CBAM [36] | 95.91 | 2.71 | 4.55 | 96.35 |
HAM | 96.03 | 2.68 | 4.28 | 96.50 |
Resnet18 | OREP | HAM | F1 | ACC | FPS |
---|---|---|---|---|---|
√ | 96.16 | 95.65 | 282 | ||
√ | √ | 96.11 | 95.86 | 338 | |
√ | √ | 96.50 | 96.03 | 250 | |
√ | √ | √ | 96.84 | 96.10 | 304 |
Method | F1 | Acc | FP | FN | FPS |
---|---|---|---|---|---|
LaneNet [17] | 94.80 | 96.38 | 7.80 | 2.44 | 44 |
SCNN [18] | 95.97 | 96.53 | 6.17 | 1.80 | 7.5 |
SAD [19] | 95.92 | 96.64 | 6.02 | 2.05 | 75 |
LaneATT [25] | 96.71 | 95.57 | 3.56 | 3.01 | 250 |
PolyLaneNet [22] | 90.62 | 93.36 | 9.42 | 9.33 | 115 |
UFLD [27] | 96.16 | 95.65 | 3.06 | 4.61 | 282 |
Ours | 96.84 | 96.10 | 2.29 | 4.00 | 304 |
Method | Normal | Crowded | Night | Noline | Shadow | Arrow | Dazzle | Curve | Cross | Total |
---|---|---|---|---|---|---|---|---|---|---|
LaneNet [17] | 82.9 | 61.1 | 53.4 | 37.7 | 56.2 | 72.2 | 54.5 | 59.3 | 5928 | 61.8 |
SCNN [18] | 90.6 | 69.7 | 66.1 | 43.4 | 66.9 | 84.1 | 58.5 | 64.4 | 1990 | 71.6 |
SAD [19] | 90.1 | 68.8 | 66.0 | 41.6 | 65.9 | 84.0 | 60.2 | 65.7 | 1998 | 70.8 |
PINet [21] | 85.8 | 67.1 | 61.7 | 44.8 | 63.1 | 79.6 | 59.4 | 63.3 | 1534 | 69.4 |
CurveLane [20] | 88.3 | 68.6 | 66.2 | 47.9 | 68.0 | 82.5 | 63.2 | 66.0 | 2817 | 71.4 |
LaneATT [25] | 91.1 | 72.9 | 68.9 | 48.3 | 70.9 | 85.4 | 65.7 | 63.3 | 1170 | 75.1 |
UFLD [27] | 91.7 | 73.0 | 70.2 | 47.2 | 74.7 | 87.6 | 64.6 | 68.7 | 1998 | 74.7 |
Ours | 92.1 | 74.1 | 71.3 | 48.4 | 77.1 | 88.3 | 63.1 | 69.3 | 1909 | 75.6 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xie, T.; Yin, M.; Zhu, X.; Sun, J.; Meng, C.; Bei, S. A Fast and Robust Lane Detection via Online Re-Parameterization and Hybrid Attention. Sensors 2023, 23, 8285. https://doi.org/10.3390/s23198285
Xie T, Yin M, Zhu X, Sun J, Meng C, Bei S. A Fast and Robust Lane Detection via Online Re-Parameterization and Hybrid Attention. Sensors. 2023; 23(19):8285. https://doi.org/10.3390/s23198285
Chicago/Turabian StyleXie, Tao, Mingfeng Yin, Xinyu Zhu, Jin Sun, Cheng Meng, and Shaoyi Bei. 2023. "A Fast and Robust Lane Detection via Online Re-Parameterization and Hybrid Attention" Sensors 23, no. 19: 8285. https://doi.org/10.3390/s23198285
APA StyleXie, T., Yin, M., Zhu, X., Sun, J., Meng, C., & Bei, S. (2023). A Fast and Robust Lane Detection via Online Re-Parameterization and Hybrid Attention. Sensors, 23(19), 8285. https://doi.org/10.3390/s23198285