[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

An enhanced asynchronous advantage actor-critic-based algorithm for performance optimization in mobile edge computing -enabled internet of vehicles networks

  • Published:
Peer-to-Peer Networking and Applications Aims and scope Submit manuscript

Abstract

Adopting Internet of Vehicles (IoV) technology has led to many new uses to improve traffic control, safety, and entertainment services. Still, the increasing amount of information these applications produce poses significant obstacles regarding response time, power usage, and other related issues. To enhance the functionality of IoV systems, this paper introduces a new way that utilizes Deep Reinforcement Learning (DRL) and Mobile Edge Computing (MEC) to improve its performance. Specifically, our DRL framework employs a convolutional neural network-based Asynchronous Advantage Actor-Critic (A3C) algorithm, chosen for its efficacy in processing spatial data relevant to IoV systems such as vehicle locations and speeds. The optimization problem considers the vehicle’s location and speed, the MEC server’s resources, and the IoV application’s requirements by formulating it as a Markov Decision Process (MDP). Utilizing the A3C approach, our Deep Neural Network (DNN) method infers an optimal offloading policy. We optimized the proposed algorithm with strategies that include adaptive learning rate, gradient clipping, entropy regularization, and generalized advantage estimation. The optimized algorithm considers factors such as distance, bandwidth, and communication requirements to provide efficient task-offloading solutions, leading to better system utility and performance. The proposed strategy outperforms comparable models through comprehensive simulations, providing an average enhancement of 20.50% in energy consumption, 15.86% in latency, and 11.94% in execution time, emphasizing the effectiveness and superiority of the suggested algorithm in dealing with various workloads while reducing energy consumption, latency, and execution times.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Availability of data and materials

In this paper, different scenarios have been used to evaluate and review the proposed algorithm and comparative algorithms, which are mentioned in the text of the paper.

References

  1. Gharehchopogh FS, Ucan A, Ibrikci T, Arasteh B, Isik G (2023) Slime mould algorithm: a comprehensive survey of its variants and applications. Arch Comput Methods Eng 1–41

  2. Minh D, Wang HX, Li YF, Nguyen TN (2022) Explainable artificial intelligence: a comprehensive review. Artif Intell Rev 1–66

  3. Moein MM et al (2022) Predictive models for concrete properties using machine learning and deep learning approaches: a review. Arch Comput Methods Eng 105444

  4. Hazarika B, Singh K, Biswas S, Li C-P (2022) DRL-based resource allocation for computation offloading in IoV networks. IEEE Trans Industr Inform 18(11):8027–8038

    Article  Google Scholar 

  5. Almutairi MS, Almutairi K, Chiroma H (2023) Hybrid of deep recurrent network and long short term memory for rear-end collision detection in fog based internet of vehicles. Expert Syst Appl 213:119033

    Article  Google Scholar 

  6. Karim A (2022) Development of secure Internet of Vehicle Things (IoVT) for smart transportation system. Electr Eng 102:108101

    Google Scholar 

  7. Cui Q et al (2022) Vehicular mobility patterns and their applications to Internet-of-Vehicles: a comprehensive survey. Sci China Inf Sci 65(11):1–42

    Article  Google Scholar 

  8. Arooj A, Farooq MS, Akram A, Iqbal R, Sharma A, Dhiman G (2022) Big data processing and analysis in internet of vehicles: architecture, taxonomy, and open research challenges. Arch Comput Methods Eng 29(2):793–829

    Article  Google Scholar 

  9. Deng T, Chen Y, Chen G, Yang M, Du L (2023) Task offloading based on edge collaboration in MEC-enabled IoV networks. J Commun Netw

  10. Sun F, Zhang Z, Zeadally S, Han G, Tong S (2022) Edge computing-enabled internet of vehicles: Towards federated learning empowered scheduling. IEEE Trans Veh Technol 71(9):10088–10103

    Article  Google Scholar 

  11. Zhang D, Cao L, Zhu H, Zhang T, Du J, Jiang K (2022) Task offloading method of edge computing in internet of vehicles based on deep reinforcement learning. Clust Comput 25(2):1175–1187

    Article  Google Scholar 

  12. Dai F, Liu G, Mo Q, Xu W, Huang B (2022) Task offloading for vehicular edge computing with edge-cloud cooperation. World Wide Web 25(5):1999–2017

    Article  Google Scholar 

  13. Moghaddasi K, Rajabi S (2023) Learning at the edge: Mobile edge computing and reinforcement learning for enhanced web application performance. In 2023 9th International Conference on Web Research (ICWR). IEEE, pp 300–304

  14. Ning Z et al (2019) Deep reinforcement learning for intelligent internet of vehicles: An energy-efficient computational offloading scheme. IEEE Trans Cognit Commun Netw 5(4):1060–1072

    Article  Google Scholar 

  15. Ju X, Su S, Xu C, Wang H (2023) Computation offloading and tasks scheduling for the internet of vehicles in edge computing: a deep reinforcement learning-based pointer network approach. Comput Netw 109572

  16. Kong X et al (2022) Deep reinforcement learning-based energy-efficient edge computing for internet of vehicles. IEEE Trans Indust Inform 18(9):6308–6316

    Article  Google Scholar 

  17. Wang K, Wang X, Liu X, Jolfaei A (2020) Task offloading strategy based on reinforcement learning computing in edge computing architecture of internet of vehicles. IEEE Access 8:173779–173789

    Article  Google Scholar 

  18. Wang J, Wang L (2021) Mobile edge computing task distribution and offloading algorithm based on deep reinforcement learning in internet of vehicles. J Ambient Intell Humaniz Comput 1–11

  19. Xu X et al (2020) Service offloading with deep Q-network for digital twinning-empowered internet of vehicles in edge computing. IEEE Trans Indust Inform 18(2):1414–1423

    Article  MathSciNet  Google Scholar 

  20. Yao L, Xu X, Bilal M, Wang H (2022) Dynamic edge computation offloading for internet of vehicles with deep reinforcement learning.  IEEE Trans Intell Transp Syst

  21. Chen Y, Zhang N, Zhang Y, Chen X (2018) Dynamic computation offloading in edge computing for internet of things. IEEE Internet Things J 6(3):4242–4251

    Article  MathSciNet  Google Scholar 

  22. Kang J, Yu R, Huang X, Zhang Y (2017) Privacy-preserved pseudonym scheme for fog computing supported internet of vehicles. IEEE Trans Intell Transp Syst 19(8):2627–2637

    Article  Google Scholar 

  23. Lee S-S, Lee S (2020) Resource allocation for vehicular fog computing using reinforcement learning combined with heuristic information. IEEE Internet Things J 7(10):10450–10464

    Article  Google Scholar 

  24. Chen M-H, Dong M, Liang B (2018) Resource sharing of a computing access point for multi-user mobile cloud offloading with delay constraints. IEEE Trans Mob Comput 17(12):2868–2881

    Article  Google Scholar 

  25. Shi J, Du J, Shen Y, Wang J, Yuan J, Han Z (2022) DRL-Based V2V computation offloading for blockchain-enabled vehicular networks. IEEE Trans Mob Comput

  26. Kabil A, Rabieh K, Kaleem F, Azer MA (2022) Vehicle to pedestrian systems: survey, challenges and recent trends. IEEE Access 10:123981–123994

    Article  Google Scholar 

  27. Tan K, Bremner D, Le Kernec J, Sambo Y, Zhang L, Imran MA (2022) Intelligent handover algorithm for vehicle-to-network communications with double-deep Q-learning. IEEE Trans Veh Technol 71(7):7848–7862

    Article  Google Scholar 

  28. Marcillo P, Tamayo-Urgilés D, Valdivieso Caraguay ÁL, Hernández-Álvarez M (2022) Security in V2I communications: a systematic literature review. Sensors 22(23):9123

    Article  Google Scholar 

  29. Yuan KH, Fang Y (2023) Which method delivers greater signal‐to‐noise ratio: Structural equation modelling or regression analysis with weighted composites? Br J Math Stat Psychol

  30. Costa LDS, Guimarães DA, Uchôa-Filho BF (2022) On the signal-to-noise ratio wall of energy detection in spectrum sensing. IEEE Access 10:16499–16511

    Article  Google Scholar 

  31. Chen C, Zeng Y, Li H, Liu Y, Wan S (2022) A multi-hop task offloading decision model in MEC-enabled internet of vehicles. IEEE Internet Things J

Download references

Funding

Authors have not received any money for presenting their paper.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Urmia Branch, Islamic Azad University, Urmia, Iran

    Komeil Moghaddasi, Shakiba Rajabi & Farhad Soleimanian Gharehchopogh

Authors
  1. Komeil Moghaddasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shakiba Rajabi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Farhad Soleimanian Gharehchopogh
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, K.M, S.R.; methodology, K.M, S.R.; software, K.M, S.R., and F.S.G; validation, F.S.G; formal analysis, K.M, S.R.; investigation, K.M, S.R.; resources, K.M and S.R.; data curation, K.M, S.R.; writing—original draft preparation, K.M, S.R.; writing—review and editing, F.S.G; visualization, K.M, S.R.; supervision, F.S.G; project administration, F.S.G; All authors have read and agreed to the published this version of the manuscript.

Corresponding author

Correspondence to Farhad Soleimanian Gharehchopogh.

Ethics declarations

Ethical approval

The authors of this paper declare that they have completely avoided publishing ethics, including plagiarism, misconduct, falsification of data, or double submission and publication, about the publication of the presented article, and there is no commercial in this regard and the authors have not received any money for presenting their paper. The corresponding author also declares that this paper has not been published elsewhere and has not been submitted to another publication at the same time.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection: 1- Track on Networking and Applications

Guest Editor: Vojislav B. Misic

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moghaddasi, K., Rajabi, S. & Gharehchopogh, F.S. An enhanced asynchronous advantage actor-critic-based algorithm for performance optimization in mobile edge computing -enabled internet of vehicles networks. Peer-to-Peer Netw. Appl. 17, 1169–1189 (2024). https://doi.org/10.1007/s12083-024-01633-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12083-024-01633-x

Keywords

Search

Navigation