More Web Proxy on the site http://driver.im/

research-article

Open access

Distilled Split Deep Neural Networks for Edge-Assisted Real-Time Systems

Authors:

Yoshitomo Matsubara,

Davide Callegaro,

Marco Levorato,

Sameer SinghAuthors Info & Claims

HotEdgeVideo'19: Proceedings of the 2019 Workshop on Hot Topics in Video Analytics and Intelligent Edges

Pages 21 - 26

https://doi.org/10.1145/3349614.3356022

Published: 07 October 2019 Publication History

Abstract

Offloading the execution of complex Deep Neural Networks (DNNs) models to compute-capable devices at the network edge, that is, edge servers, can significantly reduce capture-to-output delay. However, the communication link between the mobile devices and edge servers can become the bottleneck when channel conditions are poor. We propose a framework to split DNNs for image processing and minimize capture-to-output delay in a wide range of network conditions and computing parameters. The core idea is to split the DNN models into head and tail models, where the two sections are deployed at the mobile device and edge server, respectively. Different from prior literature presenting DNN splitting frameworks, we distill the architecture of the head DNN to reduce its computational complexity and introduce a bottleneck, thus minimizing processing load at the mobile device as well as the amount of wirelessly transferred data. Our results show 98% reduction in used bandwidth and 85% in computation load compared to straightforward splitting.

References

[1]

Alessandro Achille and Stefano Soatto. 2018. Information dropout: Learning optimal representations through noisy computation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2018).

[2]

Rohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E Dahl, and Geoffrey E Hinton. 2018. Large scale distributed neural network training through online distillation. In Sixth International Conference on Learning Representations .

[3]

Jimmy Ba and Rich Caruana. 2014. Do deep nets really need to be deep?. In Advances in neural information processing systems . 2654--2662.

[4]

Flavio Bonomi, Rodolfo Milito, Jiang Zhu, and Sateesh Addepalli. 2012. Fog computing and its role in the internet of things. In Proceedings of the first edition of the MCC workshop on Mobile cloud computing. ACM, 13--16.

Digital Library

[5]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.

[6]

Li Fei-Fei, Rob Fergus, and Pietro Perona. 2006. One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence, Vol. 28, 4 (2006), 594--611.

[7]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[8]

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely Connected Convolutional Networks. In CVPR, Vol. 1. 3.

[9]

Hyuk-Jin Jeong, InChang Jeong, Hyeon-Jae Lee, and Soo-Mook Moon. 2018. Computation Offloading for Machine Learning Web Apps in the Edge Server Environment. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS) . IEEE, 1492--1499.

[10]

Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. 2017. Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '17). ACM, New York, NY, USA, 615--629. https://doi.org/10.1145/3037697.3037698

Digital Library

[11]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Third International Conference on Learning Representations .

[12]

Nicholas D. Lane, Sourav Bhattacharya, Petko Georgiev, Claudio Forlivesi, Lei Jiao, Lorena Qendro, and Fahim Kawsar. 2016. DeepX: A Software Accelerator for Low-power Deep Learning Inference on Mobile Devices. In Proceedings of the 15th International Conference on Information Processing in Sensor Networks (IPSN '16). IEEE Press, Article 23, bibinfonumpages12 pages.

Digital Library

[13]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE, Vol. 86, 11 (1998), 2278--2324.

[14]

Jinyu Li, Rui Zhao, Jui-Ting Huang, and Yifan Gong. 2014. Learning small-size DNN with output-distribution-based criteria. In Fifteenth annual conference of the international speech communication association .

[15]

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4510--4520.

[16]

Mahadev Satyanarayanan, Victor Bahl, Ramón Caceres, and Nigel Davies. 2009. The case for vm-based cloudlets in mobile computing. IEEE pervasive Computing (2009).

[17]

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2818--2826.

[18]

Gregor Urban, Krzysztof J Geras, Samira Ebrahimi Kahou, Ozlem Aslan, Shengjie Wang, Rich Caruana, Abdelrahman Mohamed, Matthai Philipose, and Matt Richardson. 2017. Do deep convolutional nets really need to be deep and convolutional?. In Fifth International Conference on Learning Representations .

Cited By

Iyengar RDong QNguyen CPillai PSatyanarayanan M(2024)Offload Shaping for Wearable Cognitive AssistanceElectronics10.3390/electronics1320408313:20(4083)Online publication date: 17-Oct-2024
https://doi.org/10.3390/electronics13204083
Capogrosso LFraccaroli EChakraborty SFummi FCristani MDe V(2024)MTL-Split: Multi-Task Learning for Edge Devices using Split ComputingProceedings of the 61st ACM/IEEE Design Automation Conference10.1145/3649329.3655686(1-6)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3649329.3655686
Hojjat AHaberer JZainab TLandsiedel OOkoshi TKo JLiKamWa R(2024)LimitNet: Progressive, Content-Aware Image Offloading for Extremely Weak Devices & NetworksProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661856(519-533)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661856
Show More Cited By

Index Terms

Distilled Split Deep Neural Networks for Edge-Assisted Real-Time Systems
1. Information systems
  1. Information systems applications
    1. Mobile information processing systems
    2. Multimedia information systems
      1. Multimedia streaming

Recommendations

Deep Elman recurrent neural networks for statistical parametric speech synthesis

Owing to the success of deep learning techniques in automatic speech recognition, deep neural networks (DNNs) have been used as acoustic models for statistical parametric speech synthesis (SPSS). DNNs do not inherently model the temporal structure in ...
Symmetric Power Activation Functions for Deep Neural Networks
LOPAL '18: Proceedings of the International Conference on Learning and Optimization Algorithms: Theory and Applications

Common nonlinear activation functions with large saturation regions, like Sigmoid and Tanh, used for Deep Neural Networks (DNNs) can not guarantee useful and efficient training since they suffer from vanishing gradients problem. Rectified Linear Units ...
Performance comparison of CNN, QNN and BNN deep neural networks for real-time object detection using ZYNQ FPGA node
Abstract
In this manuscript, previously trained Convolutional neural network (CNN), Quantum Neural Network (QNN), and Binarized Neural Network (BNN) models performed employing Tensor Flow's Application Programming Interface (API) for real-time ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

HotEdgeVideo'19: Proceedings of the 2019 Workshop on Hot Topics in Video Analytics and Intelligent Edges

October 2019

50 pages

ISBN:9781450369282

DOI:10.1145/3349614

General Chairs:
Ganesh Ananthanarayanan
Microsoft Research, USA
,
Yunxin Liu
Microsoft Research, China
,
Yuanchao Shu
Microsoft Research, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Defense Advanced Research Projects Agency

Conference

MobiCom '19

Sponsor:

SIGMOBILE

MobiCom '19: The 25th Annual International Conference on Mobile Computing and Networking

October 21, 2019

Los Cabos, Mexico

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

58
Total Citations
View Citations
2,197
Total Downloads

Downloads (Last 12 months)419
Downloads (Last 6 weeks)43

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Iyengar RDong QNguyen CPillai PSatyanarayanan M(2024)Offload Shaping for Wearable Cognitive AssistanceElectronics10.3390/electronics1320408313:20(4083)Online publication date: 17-Oct-2024
https://doi.org/10.3390/electronics13204083
Capogrosso LFraccaroli EChakraborty SFummi FCristani MDe V(2024)MTL-Split: Multi-Task Learning for Edge Devices using Split ComputingProceedings of the 61st ACM/IEEE Design Automation Conference10.1145/3649329.3655686(1-6)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3649329.3655686
Hojjat AHaberer JZainab TLandsiedel OOkoshi TKo JLiKamWa R(2024)LimitNet: Progressive, Content-Aware Image Offloading for Extremely Weak Devices & NetworksProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661856(519-533)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661856
Li CJu PEryilmaz AShroff NPapavassiliou SSchmid S(2024)Efficient Multi-dimensional Compression for Network-edge ClassificationProceedings of the Twenty-fifth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing10.1145/3641512.3686390(91-100)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3641512.3686390
Malekzadeh MKawsar FXu CDavies N(2024)Salted Inference: Enhancing Privacy while Maintaining Efficiency of Split Inference in Mobile ComputingProceedings of the 25th International Workshop on Mobile Computing Systems and Applications10.1145/3638550.3641131(14-20)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3638550.3641131
Qin MSun CHofmann JVucinic D(2024)DISCO: Distributed Inference with Sparse Communications2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00242(2421-2429)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00242
Furutanpey ARaith PDustdar S(2024)FrankenSplit: Efficient Neural Feature Compression With Shallow Variational Bottleneck Injection for Mobile Edge ComputingIEEE Transactions on Mobile Computing10.1109/TMC.2024.338195223:12(10770-10786)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3381952
Capogrosso LXu SFraccaroli ECristani MFummi FChakraborty S(2024)Learning-Enabled CPS for Edge-Cloud Computing2024 IEEE 14th International Symposium on Industrial Embedded Systems (SIES)10.1109/SIES62473.2024.10767956(132-139)Online publication date: 23-Oct-2024
https://doi.org/10.1109/SIES62473.2024.10767956
Bove FBedogni L(2024)Smart Split: Leveraging TinyML and Split Computing for Efficient Edge AI2024 IEEE/ACM Symposium on Edge Computing (SEC)10.1109/SEC62691.2024.00052(456-460)Online publication date: 4-Dec-2024
https://doi.org/10.1109/SEC62691.2024.00052
Qiu JWang RHu BGuérin RLu C(2024)Optimizing Edge Offloading Decisions for Object Detection2024 IEEE/ACM Symposium on Edge Computing (SEC)10.1109/SEC62691.2024.00021(164-177)Online publication date: 4-Dec-2024
https://doi.org/10.1109/SEC62691.2024.00021
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents