More Web Proxy on the site http://driver.im/

research-article

Deep reinforcement learning empowered multiple UAVs-assisted caching and offloading optimization in D2D wireless networks

Authors:

Liang ZhaoAuthors Info & Claims

CF '22: Proceedings of the 19th ACM International Conference on Computing Frontiers

Pages 150 - 158

https://doi.org/10.1145/3528416.3530239

Published: 17 May 2022 Publication History

Abstract

Device-to-device (D2D) content caching is a promising technology to mitigate the backhaul pressure, and reduce the contents transmission delay. In this paper, to improve the content hit rate (CHR) and the utilization efficiency of the limited caching capacity, we put forward a caching content placement strategy by predicting the user preference and the content popularity, where unmanned aerial vehicles (UAVs) are introduced into the D2D networks to provide computation offloading services to the users. A <u>d</u>ynamic <u>r</u>esource <u>a</u>llocation <u>o</u>ptimization <u>a</u>lgorithm (DRAOA) is proposed to deploy UAVs and plan UAVs trajectory adaptively according to the users' task requirements. Simulation results show that the proposed caching content placement policy outperforms the existing baselines. Additionally, the DRAOA can effectively improve the network capacity and mitigate the computation delay compared to the other two DRL algorithms.

References

[1]

Akram Al-Hourani, Sithamparanathan Kandeepan, and Simon Lardner. 2014. Optimal LAP Altitude for Maximum Coverage. IEEE Wireless Communications Letters 3, 6 (2014), 569--572.

[2]

Tong Bai, Jingjing Wang, Yong Ren, and Lajos Hanzo. 2019. Energy-Efficient Computation Offloading for Secure UAV-Edge-Computing Systems. IEEE Transactions on Vehicular Technology 68, 6 (2019), 6074--6087.

[3]

Petros S. Bithas, Viktor Nikolaidis, Athanasios G. Kanatas, and George K. Karagiannidis. 2020. UAV-to-Ground Communications: Channel Modeling and UAV Selection. IEEE Transactions on Communications 68, 8 (2020), 5135--5144.

[4]

Mingjie Feng and Marwan Krunz. 2021. Program Placement Optimization for Storage-constrained Mobile Edge Computing Systems: A Multi-armed Bandit Approach. In 2021 IEEE 22nd International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM). 149--158.

[5]

Amal Feriani and Ekram Hossain. 2021. Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial. IEEE Communications Surveys Tutorials 23, 2 (2021), 1226--1252.

[6]

Lav Gupta, Raj Jain, and Gabor Vaszkun. 2016. Survey of Important Issues in UAV Communication Networks. IEEE Communications Surveys Tutorials 18, 2 (2016), 1123--1152.

Digital Library

[7]

Jiequ Ji, Kun Zhu, Dusit Niyato, and Ran Wang. 2020. Joint Cache Placement, Flight Trajectory, and Transmission Power Optimization for Multi-UAV Assisted Wireless Networks. IEEE Transactions on Wireless Communications 19, 8 (2020), 5389--5403.

[8]

Jiequ Ji, Kun Zhu, Dusit Niyato, and Ran Wang. 2021. Joint Trajectory Design and Resource Allocation for Secure Transmission in Cache-Enabled UAV-Relaying Networks With D2D Communications. IEEE Internet of Things Journal 8, 3 (2021), 1557--1571.

[9]

Jiequ Ji, Kun Zhu, Changyan Yi, and Dusit Niyato. 2021. Energy Consumption Minimization in UAV-Assisted Mobile-Edge Computing Systems: Joint Resource Allocation and Trajectory Design. IEEE Internet of Things Journal 8, 10 (2021), 8570--8584.

[10]

Wei Jiang, Bin Han, Mohammad Asif Habibi, and Hans Dieter Schotten. 2021. The Road Towards 6G: A Comprehensive Survey. IEEE Open Journal of the Communications Society 2 (2021), 334--366.

[11]

Yanxiang Jiang, Miaoli Ma, Mehdi Bennis, Fu-Chun Zheng, and Xiaohu You. 2019. User Preference Learning-Based Edge Caching for Fog Radio Access Network. IEEE Transactions on Communications 67, 2 (2019), 1268--1283.

[12]

Wei-Kuang Lai, You-Chiun Wang, He-Cian Lin, and Jian-Wen Li. 2020. Efficient Resource Allocation and Power Control for LTE-A D2D Communication With Pure D2D Model. IEEE Transactions on Vehicular Technology 69, 3 (2020), 3202--3216.

[13]

Xujie Li, Jichang Shen, Ying Sun, Ziya Wang, and Xuedong Zheng. 2020. A Smart Content Caching and Replacement Scheme for UAV-Assisted Fog Computing Network. In 2020 International Conference on Wireless Communications and Signal Processing (WCSP). 1040--1045.

[14]

Junyan Liu, Dapeng Li, and Youyun Xu. 2020. Collaborative Online Edge Caching With Bayesian Clustering in Wireless Networks. IEEE Internet of Things Journal 7, 2 (2020), 1548--1560.

[15]

Paolo Di Lorenzo and Gesualdo Scutari. 2016. NEXT: In-Network Nonconvex Optimization. IEEE Transactions on Signal and Information Processing over Networks 2, 2 (2016), 120--136.

[16]

Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In International conference on machine learning. PMLR, 1928--1937.

[17]

Jieying Ren and Shaoyi Xu. 2021. DDPG Based Computation Offloading and Resource Allocation for MEC Systems with Energy Harvesting. In 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring). 1--5.

[18]

Yuris Mulya Saputra, Dinh Thai Hoang, Diep N. Nguyen, and Eryk Dutkiewicz. 2021. A Novel Mobile Edge Network Architecture with Joint Caching-Delivering and Horizontal Cooperation. IEEE Transactions on Mobile Computing 20, 1 (2021), 19--31.

Digital Library

[19]

Shreshth Tuli, Shashikant Ilager, Kotagiri Ramamohanarao, and Rajkumar Buyya. 2022. Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments Using A3C Learning and Residual Recurrent Neural Networks. IEEE Transactions on Mobile Computing 21, 3 (2022), 940--954.

[20]

Duc Van Le and Chen-Khong Tham. 2018. A deep reinforcement learning based offloading scheme in ad-hoc mobile clouds. In IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). 760--765.

[21]

Dianhan Xie, Jiawei Zhang, Aimin Tang, and Xudong Wang. 2020. Multi-Dimensional Busy-Tone Arbitration for OFDMA Random Access in IEEE 802.11ax. IEEE Transactions on Wireless Communications 19, 6 (2020), 4080--4094.

[22]

Yong Zeng, Rui Zhang, and Teng Joon Lim. 2016. Wireless Communications with Unmanned Aerial Vehicles: Opportunities and Challenges. IEEE Communications Magazine 54, 5 (2016), 36--42.

Digital Library

[23]

Shangwei Zhang, Jiajia Liu, Hongzhi Guo, Mingping Qi, and Nei Kato. 2020. Envisioning Device-to-Device Communications in 6G. IEEE Network 34, 3 (2020), 86--91.

[24]

Xijian Zhong, Yan Guo, Ning Li, and Yancheng Chen. 2020. Joint Optimization of Relay Deployment, Channel Allocation, and Relay Assignment for UAVs-Aided D2D Networks. IEEE/ACM Transactions on Networking 28, 2 (2020), 804--817.

Digital Library

Cited By

Huang JZhang MWan JChen YZhang N(2024)Joint Data Caching and Computation Offloading in UAV-Assisted Internet of Vehicles via Federated Deep Reinforcement LearningIEEE Transactions on Vehicular Technology10.1109/TVT.2024.342950773:11(17644-17656)Online publication date: Nov-2024
https://doi.org/10.1109/TVT.2024.3429507

Index Terms

Deep reinforcement learning empowered multiple UAVs-assisted caching and offloading optimization in D2D wireless networks
1. Networks

Recommendations

Enhancing Throughput in Multirate Wireless Networks Using Deep Reinforcement Learning
I-DO '24: Proceedings of the 2024 International Conference on Information Technology, Data Science, and Optimization

In the wireless network environment, random backoff must be performed before packet transmission to avoid transmission collisions. Since the backoff time depends on the size of the Contention Window (CW), the setting of the contention window is crucial ...
Adaptive reservation-assisted collision resolution protocol for wireless local area networks
WCNC'09: Proceedings of the 2009 IEEE conference on Wireless Communications & Networking Conference

In conventional IEEE 802.11 medium access control protocol, the distributed coordination function is designed for the wireless stations (WSs) to perform channel contention within the wireless local area networks (WLANs). Packet collision is considered ...
Joint Downlink and Uplink Edge Computing Offloading in Ultra-Dense HetNets
Abstract
The deployment of ultra-dense heterogeneous networks (HetNets) are envisioned as the essentials to embrace of intelligent applications for the next-generation wireless networks. For ultra-dense HetNets, the small scale heterogeneous edge servers ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CF '22: Proceedings of the 19th ACM International Conference on Computing Frontiers

May 2022

321 pages

ISBN:9781450393386

DOI:10.1145/3528416

General Chair:
Luca Sterpone
Politecnico di Torino, IT
,
Program Chairs:
Andrea Bartolini
Universit`a di Bologna, IT
,
Anastasiia Butko
Lawrence Berkeley National Laboratory

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMICRO: ACM Special Interest Group on Microarchitectural Research and Processing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 May 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Liaoning Provincial Department of Education Science Foundation
Middle-aged Science and Technology Innovation Talent Support Plan of Shenyang
Natural Science Foundation of Liaoning Province
National Natural Science Foundation of China

Conference

CF '22

Sponsor:

SIGMICRO

CF '22: 19th ACM International Conference on Computing Frontiers

May 17 - 22, 2022

Turin, Italy

Acceptance Rates

Overall Acceptance Rate 273 of 785 submissions, 35%

Upcoming Conference

CF '25

Sponsor:
sigmicro

22nd ACM International Conference on Computing Frontiers

May 28 - 30, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
127
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)7

Reflects downloads up to 20 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huang JZhang MWan JChen YZhang N(2024)Joint Data Caching and Computation Offloading in UAV-Assisted Internet of Vehicles via Federated Deep Reinforcement LearningIEEE Transactions on Vehicular Technology10.1109/TVT.2024.342950773:11(17644-17656)Online publication date: Nov-2024
https://doi.org/10.1109/TVT.2024.3429507

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents