Abstract
Anticipating lane change intentions of surrounding vehicles is crucial for efficient and safe driving decision making in an autonomous driving system. Previous works often adopt physical variables such as driving speed, acceleration and so forth for lane change classification. However, physical variables do not contain semantic information. Although 3D CNNs have been developing rapidly, the number of methods utilising action recognition models and appearance feature for lane change recognition is low, and they all require additional information to pre-process data. In this work, we propose an end-to-end framework including two action recognition methods for lane change recognition, using video data collected by cameras. Our method achieves the best lane change classification results using only the RGB video data of the PREVENTION dataset. Class activation maps demonstrate that action recognition models can efficiently extract lane change motions. A method to better extract motion clues is also proposed in this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bahram, M., Hubmann, C., Lawitzky, A., Aeberhard, M., Wollherr, D.: A combined model-and learning-based framework for interaction-aware maneuver prediction. IEEE Trans. Intell. Transp. Syst. 17(6), 1538–1550 (2016)
Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the kinetics dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299–6308 (2017)
Deo, N., Trivedi, M.M.: Convolutional social pooling for vehicle trajectory prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1468–1476 (2018)
Deo, N., Trivedi, M.M.: Multi-modal trajectory prediction of surrounding vehicles with maneuver based LSTMS. In: 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 1179–1184. IEEE (2018)
Feichtenhofer, C.: X3D: expanding architectures for efficient video recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 203–213 (2020)
Feichtenhofer, C., Fan, H., Malik, J., He, K.: Slowfast networks for video recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6202–6211 (2019)
Fernández-Llorca, D., Biparva, M., Izquierdo-Gonzalo, R., Tsotsos, J.K.: Two-stream networks for lane-change prediction of surrounding vehicles. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–6. IEEE (2020)
Izquierdo, R., Parra, I., Muñoz-Bulnes, J., Fernández-Llorca, D., Sotelo, M.: Vehicle trajectory and lane change prediction using ANN and SVM classifiers. In: 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), pp. 1–6. IEEE (2017)
Izquierdo, R., Quintanar, A., Parra, I., Fernández-Llorca, D., Sotelo, M.: Experimental validation of lane-change intention prediction methodologies based on CNN and LSTM. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3657–3662. IEEE (2019)
Izquierdo, R., Quintanar, A., Parra, I., Fernández-Llorca, D., Sotelo, M.: The prevention dataset: a novel benchmark for prediction of vehicles intentions. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3114–3121. IEEE (2019)
Izquierdo, R., et al.: Vehicle lane change prediction on highways using efficient environment representation and deep learning. IEEE Access 9, 119454–119465 (2021)
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1725–1732 (2014)
Kasper, D., et al.: Object-oriented Bayesian networks for detection of lane change maneuvers. IEEE Intell. Transp. Syst. Mag. 4(3), 19–31 (2012)
Kato, S., Takeuchi, E., Ishiguro, Y., Ninomiya, Y., Takeda, K., Hamada, T.: An open approach to autonomous vehicles. IEEE Micro 35(6), 60–68 (2015). https://doi.org/10.1109/MM.2015.133
Konakalla, N., Noor, A., Singh, J.: CNN, CNN encoder-RNN decoder, and pretrained vision transformers for surrounding vehicle lane change classification at future time steps (2022). https://cs231n.stanford.edu/reports/2022/pdfs/105.pdf
Lee, D., Kwon, Y.P., McMains, S., Hedrick, J.K.: Convolution neural network-based lane change intention prediction of surrounding vehicles for ACC. In: 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), pp. 1–6. IEEE (2017)
Li, J., Lu, C., Xu, Y., Zhang, Z., Gong, J., Di, H.: Manifold learning for lane-changing behavior recognition in urban traffic. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3663–3668. IEEE (2019)
Li, J., Dai, B., Li, X., Xu, X., Liu, D.: A dynamic Bayesian network for vehicle maneuver prediction in highway driving scenarios: framework and verification. Electronics 8(1), 40 (2019)
Litman, T.: Autonomous vehicle implementation predictions. Victoria Transport Policy Institute Victoria, BC, Canada (2017)
Liu, P., Kurt, A., Özgüner, Ü.: Trajectory prediction of a lane changing vehicle based on driver behavior estimation and classification. In: 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 942–947. IEEE (2014)
Qiu, Z., Yao, T., Mei, T.: Learning spatio-temporal representation with pseudo-3D residual networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5533–5541 (2017)
Schlechtriemen, J., Wedel, A., Hillenbrand, J., Breuel, G., Kuhnert, K.D.: A lane change detection approach using feature ranking with maximized predictive power. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, pp. 108–114. IEEE (2014)
Schlechtriemen, J., Wirthmueller, F., Wedel, A., Breuel, G., Kuhnert, K.D.: When will it change the lane? A probabilistic regression approach for rarely occurring events. In: 2015 IEEE Intelligent Vehicles Symposium (IV), pp. 1373–1379. IEEE (2015)
Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)
Tran, D., Ray, J., Shou, Z., Chang, S.F., Paluri, M.: Convnet architecture search for spatiotemporal feature learning. arXiv preprint arXiv:1708.05038 (2017)
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., Paluri, M.: A closer look at spatiotemporal convolutions for action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6450–6459 (2018)
Xie, S., Sun, C., Huang, J., Tu, Z., Murphy, K.: Rethinking spatiotemporal feature learning for video understanding. arXiv preprint arXiv:1712.04851 (2017)
Yao, W., et al.: On-road vehicle trajectory collection and scene-based lane change analysis: Part II. IEEE Trans. Intell. Transp. Syst. 18(1), 206–220 (2016)
Yoon, S., Kum, D.: The multilayer perceptron approach to lateral motion prediction of surrounding vehicles for autonomous vehicles. In: 2016 IEEE Intelligent Vehicles Symposium (IV), pp. 1307–1312. IEEE (2016)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liang, K., Wang, J., Bhalerao, A. (2023). Lane Change Classification and Prediction with Action Recognition Networks. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13801. Springer, Cham. https://doi.org/10.1007/978-3-031-25056-9_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-25056-9_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25055-2
Online ISBN: 978-3-031-25056-9
eBook Packages: Computer ScienceComputer Science (R0)