More Web Proxy on the site http://driver.im/

research-article

Deep Q Network–Driven Task Offloading for Efficient Multimedia Data Analysis in Edge Computing–Assisted IoV

Authors:

Lianyong QiAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 18, Issue 2s

Article No.: 124, Pages 1 - 24

https://doi.org/10.1145/3548687

Published: 06 October 2022 Publication History

Abstract

With the prosperity of Industry 4.0, numerous emerging industries continue to gain popularity and their market scales are expanding ceaselessly. The Internet of Vehicles (IoV), one of the thriving intelligent industries, enjoys bright development prospects. However, at the same time, the reliability and availability of IoV applications are confronted with two major bottlenecks of time delay and energy consumption. To make matters worse, massive heterogeneous and multi-dimensional multimedia data generated on the IoV present a huge obstacle to effective data analysis. Fortunately, the advent of edge computing technology enables tasks to be offloaded to edge servers, which significantly reduces total overhead of IoV systems. Deep reinforcement learning (DRL), equipped with its excellent perception and decision-making capability, is undoubtedly a dominant technology to solve task offloading problems. In this article, we first employ an optimized Fuzzy C-means algorithm to cluster vehicles and other edge devices according to their respective service quality requirements. Then, we employ an election algorithm to assist in maintaining the stability of the IoV. Last, we propose a task-offloading algorithm based on the Deep Q Network (DQN) to acquire an optimal task offloading scheme. Massive simulation experiments demonstrate the superiority of our method in minimizing time delay and energy consumption.

References

[1]

Xiaokang Zhou, Xuesong Xu, Wei Liang, Zhi Zeng, and Zheng Yan. 2021. Deep-learning-enhanced multitarget detection for end-edge-cloud surveillance in smart IoT. IEEE Internet of Things Journal 8, 16 (2021), 12588–12596. DOI:

[2]

Lucas Santos Dalenogare, Guilherme Brittes Benitez, Néstor Fabián Ayala, and Alejandro Germán Frank. 2018. The expected contribution of Industry 4.0 technologies for industrial performance. International Journal of Production Economics 204 (2018), 383–394.

[3]

Hanyu Zhu, Zixin Wang, Fuqian Yang, Yong Zhou, and Xiliang Luo. 2021. Intelligent traffic network control in the era of Internet of Vehicles. IEEE Transactions on Vehicular Technology 70, 10 (2021), 9787–9802. DOI:

[4]

Kun Wang, Jun Mi, Chenhan Xu, Qingquan Zhu, Lei Shu, and Der-Jiunn Deng. 2016. Real-time load reduction in multimedia big data for mobile Internet. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 5s, Article 76 (Oct 2016), 20 pages. DOI:

Digital Library

[5]

Zheng Chang, Yang Hu, Yan Chen, and Bing Zeng. 2018. Cluster-oriented device-to-device multimedia communications: Joint power, bandwidth, and link selection optimization. IEEE Transactions on Vehicular Technology 67, 2 (2018), 1570–1581. DOI:

[6]

Xiaokang Zhou, Wei Liang, Kevin I-Kai Wang, and Laurence T. Yang. 2021. Deep correlation mining based on hierarchical hybrid networks for heterogeneous big data recommendations. IEEE Transactions on Computational Social Systems 8, 1 (2021), 171–178. DOI:

[7]

Lianyong Qi, Wenmin Lin, Xuyun Zhang, Wanchun Dou, Xialong Xu, and Jinjun Chen. 2022. A correlation graph based approach for personalized and compatible web APIs recommendation in mobile APP development. IEEE Transactions on Knowledge and Data Engineering (2022), 1–1. DOI:

[8]

Burak Kizilkaya, Enver Ever, Hakan Yekta Yatbaz, and Adnan Yazici. 2022. An effective forest fire detection framework using heterogeneous wireless multimedia sensor networks. ACM Transactions on Multimedia Computing, Communications, and Applications 18, 2, Article 47 (Feb 2022), 21 pages. DOI:

Digital Library

[9]

Xiaolong Xu, Zijie Fang, Jie Zhang, Qiang He, Dongxiao Yu, Lianyong Qi, and Wanchun Dou. 2021. Edge content caching with deep spatiotemporal residual network for IoV in smart city. ACM Transactions on Sensor Networks 17, 3, Article 29 (Jun 2021), 33 pages. DOI:

Digital Library

[10]

Zhao Tong, Feng Ye, Ming Yan, Hong Liu, and Sunitha Basodi. 2021. A survey on algorithms for intelligent computing and smart city applications. Big Data Mining and Analytics 4, 3 (2021), 155–172. DOI:

[11]

Amanpreet Kaur Sandhu. 2022. Big data with cloud computing: Discussions and challenges. Big Data Mining and Analytics 5, 1 (2022), 32–40.

[12]

Huaming Wu and Katinka Wolter. 2018. Stochastic analysis of delayed mobile offloading in heterogeneous networks. IEEE Transactions on Mobile Computing 17, 2 (2018), 461–474. DOI:

Digital Library

[13]

Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016. Edge computing: Vision and challenges. IEEE Internet of Things Journal 3, 5 (2016), 637–646. DOI:

[14]

Xiaolong Xu, Zijie Fang, Lianyong Qi, Xuyun Zhang, Qiang He, and Xiaokang Zhou. 2021. TripRes: Traffic flow prediction driven resource reservation for multimedia IoV with edge computing. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 2, Article 41 (apr 2021), 21 pages. DOI:

Digital Library

[15]

Ran Bi, Qian Liu, Jiankang Ren, and Guozhen Tan. 2021. Utility aware offloading for mobile-edge computing. Tsinghua Science and Technology 26, 2 (2021), 239–250. DOI:

[16]

Xiaolong Xu, Hao Tian, Xuyun Zhang, Lianyong Qi, Qiang He, and Wanchun Dou. 2022. DisCOV: Distributed COVID-19 detection on x-ray images with edge-cloud collaboration. IEEE Transactions on Services Computing (2022), 1–1. DOI:

[17]

Junyu Ren, Jinze Li, Huaxing Liu, and Tuanfa Qin. 2022. Task offloading strategy with emergency handling and blockchain security in SDN-empowered and fog-assisted healthcare IoT. Tsinghua Science and Technology 27, 4 (2022), 760–776. DOI:

[18]

Yue Li, Li Zhang, and Kai Zhang. 2022. IDAM: Iteratively trained deep in-loop filter with adaptive model selection. ACM Transactions on Multimedia Computing, Communications, and Applications (Mar 2022). DOI:

Digital Library

[19]

Shafiqul Islam, Abdulmotaleb El Saddik, and Anderson Sunda-Meya. 2019. Robust load frequency control for smart power grid over open distributed communication network with uncertainty. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). 4341–4346. DOI:

Digital Library

[20]

Xiaokang Zhou, Wei Liang, Weimin Li, Ke Yan, Shohei Shimizu, and Kevin I-Kai Wang. 2021. Hierarchical adversarial attacks against graph neural network based IoT network intrusion detection system. IEEE Internet of Things Journal (2021), 1–1. DOI:

[21]

Frank Höppner, Frank Klawonn, Rudolf Kruse, and Thomas Runkler. 1999. Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition. Vol. 51. John Wiley & Sons. 769–770.

[22]

Xiaolong Xu, Qinting Jiang, Peiming Zhang, Xuefei Cao, Mohammad R. Khosravi, Linss T. Alex, Lianyong Qi, and Wanchun Dou. 2022. Game theory for distributed IoV task offloading with fuzzy neural network in edge computing. IEEE Transactions on Fuzzy Systems (2022), 1–1. DOI:

[23]

H.-J. Zimmermann. 2010. Fuzzy set theory. Wiley Interdisciplinary Reviews: Computational Statistics 2, 3 (2010), 317–332.

[24]

Liu Yang, Keping Yu, Simon Xianyi Yang, Chinmay Chakraborty, Yinzhi Lu, and Tan Guo. 2021. An intelligent trust cloud management method for secure clustering in 5G enabled Internet of Medical Things. IEEE Transactions on Industrial Informatics (2021), 1–1. DOI:

[25]

Ji Li, Hui Gao, Tiejun Lv, and Yueming Lu. 2018. Deep reinforcement learning based computation offloading and resource allocation for MEC. In 2018 IEEE Wireless Communications and Networking Conference (WCNC’18). 1–6. DOI:

Digital Library

[26]

Wei Zhang, Ting Yao, Shiai Zhu, and Abdulmotaleb El Saddik. 2019. Deep learning-based multimedia analytics: A review. ACM Transactions on Multimedia Computing, Communications and Applications 15, 1s, Article 2 (Jan 2019), 26 pages. DOI:

Digital Library

[27]

Amiya Kumar Sahu, Suraj Sharma, M. Tanveer, and Rohit Raja. 2021. Internet of Things attack detection using hybrid deep learning model. Computer Communications 176 (2021), 146–154.

Digital Library

[28]

M. Tanveer, A. H. Rashid, M. A. Ganaie, M. Reza, Imran Razzak, and Kai-Lung Hua. 2022. Classification of Alzheimer’s disease using ensemble of deep neural networks trained through transfer learning. IEEE Journal of Biomedical and Health Informatics 26, 4 (2022), 1453–1463. DOI:

[29]

Zhaoliang He, Hongshan Li, Zhi Wang, Shutao Xia, and Wenwu Zhu. 2021. Adaptive compression for online computer vision: An edge reinforcement learning approach. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 4, Article 118 (Nov 2021), 23 pages. DOI:

Digital Library

[30]

Jin Wang, Jia Hu, Geyong Min, Albert Y. Zomaya, and Nektarios Georgalas. 2021. Fast adaptive task offloading in edge computing based on meta reinforcement learning. IEEE Transactions on Parallel and Distributed Systems 32, 1 (2021), 242–253. DOI:

Digital Library

[31]

James C. Bezdek, Robert Ehrlich, and William Full. 1984. FCM: The fuzzy c-means clustering algorithm. Computers & Geosciences 10, 2–3 (1984), 191–203.

[32]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.

[33]

Hado Van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double q-learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30. DOI:

[34]

Wei Liang, Yiyong Hu, Xiaokang Zhou, Yi Pan, and Kevin I-Kai Wang. 2021. Variational few-shot learning for microservice-oriented intrusion detection in distributed industrial IoT. IEEE Transactions on Industrial Informatics (2021), 1–1. DOI:

[35]

Jinming Shi, Jun Du, Jingjing Wang, Jian Wang, and Jian Yuan. 2020. Priority-aware task offloading in vehicular fog computing based on deep reinforcement learning. IEEE Transactions on Vehicular Technology 69, 12 (2020), 16067–16081. DOI:

[36]

Ming Tang and Vincent W. S. Wong. 2020. Deep reinforcement learning for task offloading in mobile edge computing systems. IEEE Transactions on Mobile Computing (2020), 1–1. DOI:

[37]

Fengqi Zhang, Xiaosong Hu, Teng Liu, Kanghui Xu, Ziwen Duan, and Hui Pang. 2021. Computationally efficient energy management for hybrid electric vehicles using model predictive control and vehicle-to-vehicle communication. IEEE Transactions on Vehicular Technology 70, 1 (2021), 237–250. DOI:

[38]

Laha Ale, Ning Zhang, Xiaojie Fang, Xianfu Chen, Shaohua Wu, and Longzhuang Li. 2021. Delay-aware and energy-efficient computation offloading in mobile-edge computing using deep reinforcement learning. IEEE Transactions on Cognitive Communications and Networking 7, 3 (2021), 881–892. DOI:

[39]

Hui Huang, Qiang Ye, and Yitong Zhou. 2021. Deadline-aware task offloading with partially-observable deep reinforcement learning for multi-access edge computing. IEEE Transactions on Network Science and Engineering (2021), 1–1. DOI:

[40]

Ting Cai, Zhihua Yang, Yufei Chen, Wuhui Chen, Zibin Zheng, Yang Yu, and Hong-Ning Dai. 2021. Cooperative data sensing and computation offloading in UAV-assisted crowdsensing with multi-agent deep reinforcement learning. IEEE Transactions on Network Science and Engineering (2021), 1–1. DOI:

[41]

Guanjin Qu, Huaming Wu, Ruidong Li, and Pengfei Jiao. 2021. DMRO: A deep meta reinforcement learning-based task offloading framework for edge-cloud computing. IEEE Transactions on Network and Service Management 18, 3 (2021), 3448–3459. DOI:

[42]

Hongchang Ke, Jian Wang, Lingyue Deng, Yuming Ge, and Hui Wang. 2020. Deep reinforcement learning-based adaptive computation offloading for MEC in heterogeneous vehicular networks. IEEE Transactions on Vehicular Technology 69, 7 (2020), 7916–7929. DOI:

[43]

Qi Qi, Jingyu Wang, Zhanyu Ma, Haifeng Sun, Yufei Cao, Lingxin Zhang, and Jianxin Liao. 2019. Knowledge-driven service offloading decision for vehicular edge computing: A deep reinforcement learning approach. IEEE Transactions on Vehicular Technology 68, 5 (2019), 4192–4203. DOI:

[44]

Xiaolong Xu, Qihe Huang, Yiwen Zhang, Shancang Li, Lianyong Qi, and Wanchun Dou. 2021. An LSH-based offloading method for IoMT services in integrated cloud-edge environment. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 3s, Article 94 (Jan 2021), 19 pages. DOI:

Digital Library

[45]

Xiaokang Zhou, Xiang Yang, Jianhua Ma, and Kevin I-Kai Wang. 2021. Energy efficient smart routing based on link correlation mining for wireless edge computing in IoT. IEEE Internet of Things Journal (2021), 1–1. DOI:

[46]

Xiuwen Yin, Jianqi Liu, Xiaochun Cheng, and Xiaoming Xiong. 2021. Large-size data distribution in IoV based on 5G/6G compatible heterogeneous network. IEEE Transactions on Intelligent Transportation Systems (2021), 1–13. DOI:

Digital Library

[47]

Xianfu Chen, Honggang Zhang, Celimuge Wu, Shiwen Mao, Yusheng Ji, and Medhi Bennis. 2018. Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet of Things Journal 6, 3 (2018), 4005–4018.

[48]

M. A. Ganaie, Minghui Hu, M. Tanveer, et al. 2021. Ensemble deep learning: A review. arXiv preprint arXiv:2104.02395.

Cited By

Liu HLi NKou HMeng SLi Q(2025)FSRPCL: Privacy-Preserve Federated Social Relationship Prediction with Contrastive LearningTsinghua Science and Technology10.26599/TST.2024.901007730:4(1762-1781)Online publication date: Aug-2025
https://doi.org/10.26599/TST.2024.9010077
Cheng YCao WFang HZang S(2025)A Context-Aware Edge-Cloud Collaboration Framework for QoS PredictionTsinghua Science and Technology10.26599/TST.2024.901002730:3(1201-1214)Online publication date: Jun-2025
https://doi.org/10.26599/TST.2024.9010027
Zhu SZhu BChi KQiu JShi HGao X(2025)Maximizing Long-Term Task Completion Ratio of UAV-Enabled Wirelessly-Powered MEC SystemsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3712599Online publication date: 20-Jan-2025
https://doi.org/10.1145/3712599
Show More Cited By

Index Terms

Deep Q Network–Driven Task Offloading for Efficient Multimedia Data Analysis in Edge Computing–Assisted IoV
1. Networks
  1. Network algorithms

Recommendations

Task offloading method of edge computing in internet of vehicles based on deep reinforcement learning
Abstract
Compared with the traditional network tasks, the emerging Internet of Vehicles (IoV) technology has higher requirements for network bandwidth and delay. However, due to the limitation of computing resources and battery capacity of existing mobile ...
Deadline-Aware Task Offloading for Vehicular Edge Computing Networks Using Traffic Light Data
As vehicles have become increasingly automated, novel vehicular applications have emerged to enhance the safety and security of the vehicles and improve user experience. This brings ever-increasing data and resource requirements for timely computation by ...
Task offloading for vehicular edge computing with edge-cloud cooperation
Abstract
Vehicular edge computing (VEC) is emerging as a novel computing paradigm to meet low latency demands for computation-intensive vehicular applications. However, most existing offloading schemes do not take the dynamic edge-cloud computing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 18, Issue 2s

June 2022

383 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3561949

Editor:
Abdulmotaleb El Saddik
Mohamed Bin Zayed University of Artificial Intelligence, UAE and University of Ottawa, Canada

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 October 2022

Online AM: 21 July 2022

Accepted: 23 June 2022

Revised: 11 May 2022

Received: 30 November 2021

Published in TOMM Volume 18, Issue 2s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

Natural Science Foundation of Jiangsu Province of China
Financial and Science Technology Plan Project of Xinjiang Production and Construction Corps
National Natural Science Foundation of China
Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
701
Total Downloads

Downloads (Last 12 months)171
Downloads (Last 6 weeks)7

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu HLi NKou HMeng SLi Q(2025)FSRPCL: Privacy-Preserve Federated Social Relationship Prediction with Contrastive LearningTsinghua Science and Technology10.26599/TST.2024.901007730:4(1762-1781)Online publication date: Aug-2025
https://doi.org/10.26599/TST.2024.9010077
Cheng YCao WFang HZang S(2025)A Context-Aware Edge-Cloud Collaboration Framework for QoS PredictionTsinghua Science and Technology10.26599/TST.2024.901002730:3(1201-1214)Online publication date: Jun-2025
https://doi.org/10.26599/TST.2024.9010027
Zhu SZhu BChi KQiu JShi HGao X(2025)Maximizing Long-Term Task Completion Ratio of UAV-Enabled Wirelessly-Powered MEC SystemsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3712599Online publication date: 20-Jan-2025
https://doi.org/10.1145/3712599
Wu HGu ALiang Y(2025)Federated Reinforcement Learning-Empowered Task Offloading for Large Models in Vehicular Edge ComputingIEEE Transactions on Vehicular Technology10.1109/TVT.2024.348187674:2(1979-1991)Online publication date: Feb-2025
https://doi.org/10.1109/TVT.2024.3481876
Hu YXu XQi LZhou XXia X(2025)Latency- and Privacy-Aware Convolutional Neural Network Distributed Inference for Reliable Artificial Intelligence SystemsIEEE Transactions on Artificial Intelligence10.1109/TAI.2024.33668806:2(365-377)Online publication date: Feb-2025
https://doi.org/10.1109/TAI.2024.3366880
Zhang PWang ETan LKumar NWang JLiu K(2025)Enhancing task offloading in vehicular networks: A multi-agent cloud-edge-device frameworkVehicular Communications10.1016/j.vehcom.2025.10089853(100898)Online publication date: Jun-2025
https://doi.org/10.1016/j.vehcom.2025.100898
Miao WZhao XZhang YChen SLi XLi Q(2024)A Deep Learning-Based Method for Preventing Data Leakage in Electric Power Industrial Internet of Things Business Data InteractionsSensors10.3390/s2413406924:13(4069)Online publication date: 22-Jun-2024
https://doi.org/10.3390/s24134069
Wang SMa XLi X(2024)A Parameter Adaptive Method for Image SmoothingTsinghua Science and Technology10.26599/TST.2023.901006829:4(1138-1151)Online publication date: Aug-2024
https://doi.org/10.26599/TST.2023.9010068
Balachander TRamana KMohana RSrivastava GGadekallu T(2024)Cooperative Spectrum Sensing Deployment for Cognitive Radio Networks for Internet of Things 5G Wireless CommunicationTsinghua Science and Technology10.26599/TST.2023.901006529:3(698-720)Online publication date: Jun-2024
https://doi.org/10.26599/TST.2023.9010065
Zhao XYuan GQiu SXu CWei S(2024)A mobile edge computing-focused transferable sensitive data identification method based on product quantizationJournal of Cloud Computing10.1186/s13677-024-00662-413:1Online publication date: 8-May-2024
https://doi.org/10.1186/s13677-024-00662-4
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents