[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

VEC Collaborative Task Offloading and Resource Allocation Based on Deep Reinforcement Learning Under Parking Assistance

  • Research
  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

With the emergence of autonomous vehicles, meeting the vehicle’s computing needs for computationally intensive and latency-sensitive tasks has become a challenge. Cellular Vehicle-to-Everything (C-V2X), an essential Internet of Vehicles technology, is expected to be enhanced and strengthened in the 6G era to improve road traffic safety and realize intelligent transportation. However, when too many computationally intensive and latency-sensitive tasks of autonomous vehicles are offloaded to the MEC server, the server can become overloaded, unable to meet the offloading demands of numerous vehicles. Inspired by the idle parked cars on both sides of the road, this paper proposes utilizing idle vehicles to assist VEC servers in offloading computing tasks, thereby increasing resource capacity and expanding communication range. Therefore, this paper firstly utilizes roadside parked vehicles with idle computing resources as a task offloading platform to develop a mobile edge computing task offloading strategy based on roadside parked vehicle collaboration. Secondly, a more flexible offloading solution is proposed based on comprehensive consideration of offloading decisions and resource allocation in a multi-user and multi-server parked vehicle-assisted MEC environment. Subsequently, to ensure service quality for end users, we consider weighted total delay cost and energy consumption as optimization objectives. The optimization problem is further formulated as a Markov decision process, and a joint computing offloading and resource allocation optimization algorithm based on deep reinforcement learning is proposed to minimize the total delay and energy consumption of vehicle users. Finally, experimental results validate the algorithm. Compared with other benchmark solutions, the proposed scheme improves system performance by 78%, 77.3%, 72.7%, and 71.1%, respectively. The proposed scheme reduces the total system cost during task offloading and enhances system performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

We are open to sharing code and supplementary materials with interested reader upon reasonable request.

References

  1. Qureshi, K. N., Alhudhaif, A., Haidar, S. W., Majeed, S., & Jeon, G. (2022). Secure data communication for wireless mobile nodes in intelligent transportation systems. Microprocessors and Microsystems, 90, 104501. https://doi.org/10.1016/j.micpro.2022.104501

    Article  Google Scholar 

  2. Liu, L., Chen, C., Pei, Q., Maharjan, S., & Zhang, Y. (2021). Vehicular edge computing and networking: A survey. Mobile Networks and Applications, 26, 1145–1168. https://doi.org/10.1007/s11036-020-01624-1

    Article  Google Scholar 

  3. Abbas, N., Zhang, Y., Taherkordi, A., & Skeie, T. (2017). Mobile edge computing: A survey. IEEE Internet of Things Journal, 5(1), 450–465. https://doi.org/10.1109/JIOT.2017.2750180

    Article  Google Scholar 

  4. Ouyang, T., Zhou, Z., & Chen, X. (2018). Follow me at the edge: Mobility-aware dynamic service placement for mobile edge computing. IEEE Journal on Selected Areas in Communications, 36(10), 2333–2345. https://doi.org/10.1109/JSAC.2018.2869954

    Article  Google Scholar 

  5. Mohammed, B., Hamdan, M., Bassi, J. S., Jamil, H. A., Khan, S., Elhigazi, A., & Marso-no, M. N. (2020). Edge computing intelligence using robust feature selection for network traffic classification in Internet-of-Things. IEEE Access, 8, 224059–224070. https://doi.org/10.1109/ACCESS.2020.3037492

    Article  Google Scholar 

  6. Al-Mayouf, Y. R. B., Abdullah, N. F., Mahdi, O. A., Khan, S., Ismail, M., Guizani, M., & Ahmed, S. H. (2018). Real-time intersection-based segment aware routing algorithm for urban vehicular networks. IEEE Transactions on Intelligent Transportation Systems, 19(7), 2125–2141. https://doi.org/10.1109/TITS.2018.2823312

    Article  Google Scholar 

  7. Xu, J., Chen, L., & Zhou, P. (2018). Joint service caching and task offloading for mobile edge computing in dense networks. In IEEE INFOCOM 2018-IEEE conference on computer communications (pp 207–215). https://doi.org/10.1109/INFOCOM.2018.8485977

  8. Zhou, Z., Yu, H., Xu, C., Xiong, F., Jia, Y., & Li, G. (2017). Joint relay selection and spectrum allocation in D2D-based cooperative vehicular networks. In 2017 International conference on information and communication technology convergence (ICTC) (pp. 241–246). https://doi.org/10.1109/ICTC.2017.8190978

  9. Liu, J., Wan, J., Zeng, B., Wang, Q., Song, H., & Qiu, M. (2017). A scalable and quick-response software defined vehicular network assisted by mobile edge computing. IEEE Communications Magazine, 55(7), 94–100. https://doi.org/10.1109/MCOM.2017.1601150

    Article  Google Scholar 

  10. Lu, J., Jiang, J., Balasubramanian, V., Khosravi, M. R., & Xu, X. (2022). Deep reinforcement learning-based multi-objective edge server placement in Internet of Vehicles. Computer Communications, 187, 172–180. https://doi.org/10.1016/j.comcom.2022.02.011

    Article  Google Scholar 

  11. Nguyen, K., Drew, S., Huang, C., & Zhou, J. (2021). EdgePV: Collaborative edge computing framework for task offloading. In ICC 2021-IEEE international conference on communications (pp. 1–6). https://doi.org/10.1109/ICC42927.2021.9500400

  12. Qi, W., Li, Q., Song, Q., Guo, L., & Jamalipour, A. (2021). Extensive edge intelligence for future vehicular networks in 6G. IEEE Wireless Communications, 28(4), 128–135. https://doi.org/10.1109/MWC.001.2000393

    Article  Google Scholar 

  13. Rahman, F. H., Iqbal, A. Y. M., Newaz, S. S., Wan, A. T., & Ahsan, M. S. (2019). Street parked vehicles based vehicular fog computing: TCP throughput evaluation and future research direction. In 2019 21st International conference on advanced communication technology (ICACT) (pp. 26–31). https://doi.org/10.23919/ICACT.2019.8701912

  14. Al-Mayouf, Y. R. B., Ismail, M., Abdullah, N. F., Wahab, A. W. A., Mahdi, O. A., Khan, S., & Choo, K. K. R. (2016). Efficient and stable routing algorithm based on user mobility and node density in urban vehicular network. PLoS ONE, 11(11), e0165966. https://doi.org/10.1371/journal.pone.0165966

    Article  Google Scholar 

  15. Guo, K., Gao, R., Xia, W., & Quek, T. Q. (2020). Online learning based computation offloading in MEC systems with communication and computation dynamics. IEEE Transactionson Communications, 69(2), 1147–1162. https://doi.org/10.1109/TCOMM.2020.3038875

    Article  Google Scholar 

  16. Liu, Q., Han, T., & Ansari, N. (2018). Joint radio and computation resource management for low latency mobile edge computing. In 2018 IEEE global communications conference (GLOBECOM) (pp. 1–7). https://doi.org/10.1109/GLOCOM.2018.8647792

  17. Saleem, U., Liu, Y., Jangsher, S., Li, Y., & Jiang, T. (2020). Mobility-aware joint task scheduling and resource allocation for cooperative mobile edge computing. IEEE Transactions on Wireless Communications, 20(1), 360–374. https://doi.org/10.1109/TWC.2020.3024538

    Article  Google Scholar 

  18. Wang, G., Xu, F., & Zhao, C. (2020). Multi-access edge computing based vehicular network: Joint task scheduling and resource allocation strategy. In 2020 IEEE international conference on communications workshops (ICC Workshops) (pp.1–6). https://doi.org/10.1109/ICCWorkshops49005.2020.9145277

  19. Zhang, C., & Zheng, Z. (2019). Task migration for mobile edge computing using deep reinforcement learning. Future Generation Computer Systems, 96, 111–118. https://doi.org/10.1016/j.future.2019.01.059

    Article  Google Scholar 

  20. Huang, B., Li, Z., Xu, Y., Pan, L., Wang, S., Hu, H., & Chang, V. (2020). Deep reinforcement learning for performance-aware adaptive resource allocation in mobile edge computing. Wireless Communications and Mobile Computing, 2020, 1–17. https://doi.org/10.1155/2020/2765491

    Article  Google Scholar 

  21. Ma, S., Guo, S., Wang, K., Jia, W., & Guo, M. (2020). A cyclic game for service-oriented resource allocation in edge computing. IEEE Transactions on Services Computing, 13(4), 723–734. https://doi.org/10.1109/TSC.2020.2966196

    Article  Google Scholar 

  22. Du, T., Zhu, J., Liu, N., Cao, K., & Guo. (2022). Parking edge computing: Task offloading based on roadside parking in vehicle ad hoc networks. Journal of Chinese Computer Systems, 43(2), 416–421.

    Google Scholar 

  23. Du, J., Yu, F. R., Chu, X., Feng, J., & Lu, G. (2018). Computation offloading and resource allocation in vehicular networks based on dual-side cost minimization. IEEE Transactions onVehicular Technology, 68(2), 1079–1092. https://doi.org/10.1109/TVT.2018.2883156

    Article  Google Scholar 

  24. Ren, J., Yu, G., Cai, Y., He, Y., & Qu, F. (2017). Partial offloading for latency minimization in mobile-edge computing. In GLOBECOM 2017—2017 IEEE global communications conference (pp. 1–6). https://doi.org/10.1109/GLOCOM.2017.8254550

  25. You, C., Huang, K., Chae, H., & Kim, B. H. (2016). Energy-efficient resource allocation for mobile-edge computation offloading. IEEE Transactions on Wireless Communications, 16(3), 1397–1411. https://doi.org/10.1109/TWC.2016.2633522

    Article  Google Scholar 

  26. Mao, Y., Zhang, J., & Letaief, K. B. (2017). Joint task offloading scheduling and transmit power allocation for mobile-edge computing systems. In 2017 IEEE wireless communications and networking conference (WCNC) (pp. 1–6). https://doi.org/10.1109/WCNC.2017.7925615

  27. Liu, L., Chang, Z., Guo, X., & Ristaniemi, T. (2017). Multi-objective optimization for computation offloading in mobile-edge computing. In 2017 IEEE symposium on computers and communications (ISCC) (pp. 832–837). https://doi.org/10.1109/ISCC.2017.8024630

  28. Tran, T. X., & Pompili, D. (2018). Joint task offloading and resource allocation for multi-server mobile-edge computing networks. IEEE Transactions on Vehicular Technology, 68(1), 856–868. https://doi.org/10.1109/TVT.2018.2881191

    Article  Google Scholar 

  29. Li, B., Hou, F., Ding, H., & Wu, H. (2021). Community based parking: Finding and predicting available parking spaces based on the Internet of Things and crowdsensing. Computers & Industrial Engineering, 162, 107755. https://doi.org/10.1016/j.cie.2021.107755

    Article  Google Scholar 

  30. Qu, X., Ong, Y. S., Hou, Y., & Shen, X. (2019). Memetic evolution strategy for reinforcement learning. In 2019 IEEE congress on evolutionary computation (CEC) (pp. 1922–1928). https://doi.org/10.1109/CEC.2019.8789935

  31. Tong, Z., Deng, X., Chen, H., Mei, J., & Liu, H. (2020). QL-HEFT: A novel machine learning scheduling scheme base on cloud computing environment. Neural Computing and Applications, 32, 5553–5570. https://doi.org/10.1007/s00521-019-04118-8

    Article  Google Scholar 

  32. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602. https://doi.org/10.48550/arXiv.1312.5602

  33. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., & Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533. https://doi.org/10.1038/nature14236

    Article  Google Scholar 

  34. Yang, J., Lin, F., Saini, D. K., Zhu, Y., Li, Y., & Guo, Z. (2022). Energy-efficient computation offloading and resource allocation in delay-constrained vehicular edge network. International Journal of Communication Systems. https://doi.org/10.1002/dac.5335

    Article  Google Scholar 

Download references

Funding

Science and Technology Program of Gansu Province, 23YFGA0062.

Author information

Authors and Affiliations

  1. School of Computer and Communication, Lanzhou University of Technology, Lanzhou, 730050, China

    Jianbin Xue, Fei Shao, Tingjuan Zhang, Guiying Tian & Hengjie Jiang

Authors
  1. Jianbin Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tingjuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guiying Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hengjie Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XJB and SF conceived the idea and completed the theoretical analysis. ZTJ and TGY completed the numerical simulation. JHJ has improved the system model and algorithm of this paper. All authors had useful discussions and reviewed the manuscript. The author reads and approves the final manuscript.

Corresponding author

Correspondence to Fei Shao.

Ethics declarations

Conflict of interest

The authors assert that they do not have any apparent competing financial interests or personal relationships that could not have influenced the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, J., Shao, F., Zhang, T. et al. VEC Collaborative Task Offloading and Resource Allocation Based on Deep Reinforcement Learning Under Parking Assistance. Wireless Pers Commun 136, 321–345 (2024). https://doi.org/10.1007/s11277-024-11258-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-024-11258-1

Keywords

Search

Navigation