[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

Task migration for mobile edge computing using deep reinforcement learning

Published: 01 July 2019 Publication History

Abstract

Mobile edge computing (MEC) is a new network architecture that puts computing capabilities and storage resource at the edges of the network in a distributed manner, instead of a kind of centralized cloud computing architecture. The computation tasks of the users can be offloaded to the nearby MEC servers to achieve high quality of computation experience. As many applications’ users have high mobility, such as applications of autonomous driving, the original MEC server with the offloaded tasks may become far from the users. Therefore, the key challenge of the MEC is to make decisions on where and when the tasks had better be migrated according to users’ mobility. Existing works formulated this problem as a sequential decision making model and using Markov decision process (MDP) to solve, with assumption that mobility pattern of the users is known ahead. However, it is difficult to get users’ mobility pattern in advance. In this paper, we propose a deep Q-network (DQN) based technique for task migration in MEC system. It can learn the optimal task migration policy from previous experiences without necessarily acquiring the information about users’ mobility pattern in advance. Our proposed task migration algorithm is validated by conducting extensive simulations in the MEC system.

Highlights

Mobile edging computing (MEC) is an effective way to reduce the computation time for users.
Task migration is necessary for high mobility users.
Deep reinforcement learning is effective for task migration in MEC.

References

[1]
Zhang Q., Cheng L., Boutaba R., Cloud computing: state-of-the-art and research challenges, J. Internet Serv. Appl. 1 (1) (2010) 7–18,.
[2]
Zhang Y., Qiu M., Tsai C., Hassan M.M., Alamri A., Health-CPS: Healthcare cyber-physical system assisted by cloud and big data, IEEE Syst. J. 11 (1) (2017) 88–95,.
[3]
Abbas N., Zhang Y., Taherkordi A., Skeie T., Mobile edge computing: A survey, IEEE Internet Things J. 5 (1) (2018) 450–465,.
[4]
Dinh H.T., Lee C., Niyato D., Wang P., A survey of mobile cloud computing: architecture, applications, and approaches, Wireless Commun. Mobile Comput. 13 (18) (2011) 1587–1611,.
[5]
Mach P., Becvar Z., Mobile edge computing: A survey on architecture and computation offloading, IEEE Commun. Surv. Tutor. 19 (3) (2017) 1628–1656,.
[6]
Ksentini A., Taleb T., Chen M., A Markov decision process-based service migration procedure for follow me cloud, in: 2014 IEEE International Conference on Communications (ICC), 2014, pp. 1350–1354,.
[7]
Wang S., Urgaonkar R., He T., Zafer M., Chan K., Leung K.K., Mobility-induced service migration in mobile micro-clouds, in: 2014 IEEE Military Communications Conference, 2014, pp. 835–840,.
[8]
Nadembega A., Hafid A.S., Brisebois R., Mobility prediction model-based service migration procedure for follow me cloud to support qos and qoe, in: 2016 IEEE International Conference on Communications (ICC), 2016, pp. 1–6,.
[9]
Wang S., Urgaonkar R., Chan K., He T., Zafer M., Leung K.K., Dynamic service placement for mobile micro-clouds with predicted future costs, in: 2015 IEEE International Conference on Communications (ICC), 2015, pp. 5504–5510,.
[10]
Urgaonkar R., Wang S., He T., Zafer M., Chan K., Leung K.K., Dynamic service migration and workload scheduling in edge-clouds, Perform. Eval. 91 (2015) 205–228,. special Issue: Performance 2015.
[11]
Secci S., Raad P., Gallard P., Linking virtual machine mobility to user mobility, IEEE Trans. Netw. Serv. Manag. 13 (4) (2016) 927–940.
[12]
Taleb T., Ksentini A., An analytical model for follow me cloud, in: 2013 IEEE Global Communications Conference (GLOBECOM), 2013, pp. 1291–1296,.
[13]
Taleb T., Ksentini A., Frangoudis P., Follow-me cloud: When cloud services follow mobile users, IEEE Trans. Cloud Comput. (2018),. 1–1.
[14]
Sun X., Ansari N., Primal: PRofit Maximization Avatar pLacement for mobile edge computing, in: 2016 IEEE International Conference on Communications (ICC), 2016, pp. 1–6,.
[15]
Wang S., Urgaonkar R., He T., Chan K., Zafer M., Leung K.K., Dynamic service placement for mobile micro-clouds with predicted future costs, IEEE Trans. Parallel Distrib. Syst. 28 (4) (2017) 1002–1016,.
[16]
Ha K., Abe Y., Chen Z., Hu W., Amos B., Pillai P., Satyanarayanan M., Adaptive VM Handoff Across Cloudlets, 2015.
[17]
D. Farinacci, V. Fuller, D. Meyer, D. Lewis, The Locator/ID Separation Protocol (LISP), RFC 6830 (Jan. 2013). https://doi.org/10.17487/RFC6830. URL https://rfc-editor.org/rfc/rfc6830.txt.
[18]
Qiu M., Sha E.H.M., Cost minimization while satisfying hard/soft timing constraints for heterogeneous embedded systems, ACM Trans. Des. Autom. Electron. Syst. 14 (2) (2009) 25:1–25:30,.
[19]
Zhu X., Qin X., Qiu M., Qos-aware fault-tolerant scheduling for real-time tasks on heterogeneous clusters, IEEE Trans. Comput. 60 (6) (2011) 800–812,.
[20]
Qiu M., Ming Z., Li J., Gai K., Zong Z., Phase-change memory optimization for green cloud with genetic algorithm, IEEE Trans. Comput. 64 (12) (2015) 3528–3540,.
[21]
Mnih V., Kavukcuoglu K., Silver D., Rusu A.A., Veness J., Bellemare M.G., Graves A., Riedmiller M., Fidjeland A.K., Ostrovski G., Petersen S., Beattie C., Sadik A., Antonoglou I., King H., Kumaran D., Wierstra D., Legg S., Hassabis D., Human-level control through deep reinforcement learning, Nature 518 (7540) (2015) 529–533,.
[22]
Sutton P.S., Barto A.G., Reinforcement Learning: An Introduction, MIT Press, 1998.
[23]
LeCun Y., Bengio Y., Hinton G., Deep learning, Nature 521 (7553) (2015) 436–444,.
[24]
Bellman R., Dynamic Programming, Princeton University Press, 1957.
[25]
Zhang C., Gu B., Liu Z., Yamori K., Tanaka Y., Cost- and energy-aware multi-flow mobile data offloading using Markov decision process, IEICE Trans. Commun. E101-B (3) (2018),.
[26]
Zhang C., Gu B., Liu Z., Yamori K., Tanaka Y., A reinforcement learning approach for cost- and energy-aware mobile data offloading, in: Proc. 16th Asia-Pacific Network Operations and Management Symposium (APNOMS 2016), 2016, pp. 1–6,.
[27]
Gai K., Qiu M., Blend arithmetic operations on tensor-based fully homomorphic encryption over real numbers, IEEE Trans. Ind. Inf. 14 (8) (2018) 3590–3598,.

Cited By

View all
  • (2024)Tconns: a novel time-varying context-aware offloading strategy for mobile edge computingEURASIP Journal on Wireless Communications and Networking10.1186/s13638-023-02331-72024:1Online publication date: 4-Jan-2024
  • (2024)SMART: Cost-Aware Service Migration Path Selection Based on Deep Reinforcement LearningIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.337892025:9(12421-12436)Online publication date: 5-Apr-2024
  • (2024)To Migrate or Not to Migrate: An Analysis of Operator Migration in Distributed Stream ProcessingIEEE Communications Surveys & Tutorials10.1109/COMST.2023.333095326:1(670-705)Online publication date: 1-Jan-2024
  • Show More Cited By

Index Terms

  1. Task migration for mobile edge computing using deep reinforcement learning
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Please enable JavaScript to view thecomments powered by Disqus.

          Information & Contributors

          Information

          Published In

          cover image Future Generation Computer Systems
          Future Generation Computer Systems  Volume 96, Issue C
          Jul 2019
          750 pages

          Publisher

          Elsevier Science Publishers B. V.

          Netherlands

          Publication History

          Published: 01 July 2019

          Author Tags

          1. 00-01
          2. 99-00

          Author Tags

          1. Service migration
          2. Mobile edge computing
          3. Deep reinforcement learning

          Qualifiers

          • Research-article

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)0
          • Downloads (Last 6 weeks)0
          Reflects downloads up to 22 Dec 2024

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)Tconns: a novel time-varying context-aware offloading strategy for mobile edge computingEURASIP Journal on Wireless Communications and Networking10.1186/s13638-023-02331-72024:1Online publication date: 4-Jan-2024
          • (2024)SMART: Cost-Aware Service Migration Path Selection Based on Deep Reinforcement LearningIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.337892025:9(12421-12436)Online publication date: 5-Apr-2024
          • (2024)To Migrate or Not to Migrate: An Analysis of Operator Migration in Distributed Stream ProcessingIEEE Communications Surveys & Tutorials10.1109/COMST.2023.333095326:1(670-705)Online publication date: 1-Jan-2024
          • (2024)Computational task offloading algorithm based on deep reinforcement learning and multi-task dependencyTheoretical Computer Science10.1016/j.tcs.2024.114462993:COnline publication date: 27-Apr-2024
          • (2024)Mobility-aware task offloading in MEC with task migration and result cachingAd Hoc Networks10.1016/j.adhoc.2024.103411156:COnline publication date: 1-Apr-2024
          • (2024)Multi-user reinforcement learning based task migration in mobile edge computingFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-023-1346-318:4Online publication date: 1-Aug-2024
          • (2024)VEC Collaborative Task Offloading and Resource Allocation Based on Deep Reinforcement Learning Under Parking AssistanceWireless Personal Communications: An International Journal10.1007/s11277-024-11258-1136:1(321-345)Online publication date: 1-May-2024
          • (2024)Energy-efficient secure dynamic service migration for edge-based 3-D networksTelecommunications Systems10.1007/s11235-023-01094-285:3(477-490)Online publication date: 1-Mar-2024
          • (2023)Task Migration Optimization Algorithm in Mobile Edge ComputingProceedings of the 2023 2nd International Conference on Networks, Communications and Information Technology10.1145/3605801.3605827(133-137)Online publication date: 16-Jun-2023
          • (2023)Resource Management in Mobile Edge Computing: A Comprehensive SurveyACM Computing Surveys10.1145/358963955:13s(1-37)Online publication date: 13-Jul-2023
          • Show More Cited By

          View Options

          View options

          Media

          Figures

          Other

          Tables

          Share

          Share

          Share this Publication link

          Share on social media