Poster Abstract: Efficient Knowledge Distillation to Train Lightweight Neural Network for Heterogeneous Edge Devices
Pages 546 - 547
Abstract
This poster presents a novel approach that harnesses large-sized deep neural networks to craft lightweight variants, addressing constraints in storage, processing speed, and task execution time on heterogeneous edge devices. Knowledge distillation is employed to refine the training of lightweight deep neural networks, and a novel early termination technique is introduced to optimize resource utilization and expedite the training process. This approach yields satisfactory accuracy while accommodating diverse heterogeneous edge device constraints.
References
[1]
V. M. Janakiraman. 2018. Explaining Aviation Safety Incidents using Deep Temporal Multiple Instance Learning. In Proc. ACM SIGKDD. 406--415.
[2]
A. Mishra and D. Marr. 2018. Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy. In Proc. ICLR. 1--17.
[3]
R. Mishra, A. Gupta, H. P. Gupta, and T. Dutta. 2022. A Sensors Based Deep Learning Model for Unseen Locomotion Mode Identification using Multiple Semantic Matrices. IEEE Trans. Mobile Comput. 21, 3 (2022), 799--810.
[4]
H. Xue, W. Jiang, C. Miao, Y. Yuan, F. Ma, X. Ma, Y. Wang, S. Yao, W. Xu, A. Zhang, et al. 2019. DeepFusion: A Deep Learning Framework for the Fusion of Heterogeneous Sensory Data. In Proc. ACM Sensys. 151--160.
[5]
S. Yao, S. Hu, Y. Zhao, A. Zhang, and T. Abdelzaher. 2017. Deepsense: A Unified Deep Learning Framework for Time-series Mobile Sensing Data Processing. In Proc. WWW. 351--360.
[6]
H. Zhao, X. Sun, J. Dong, C. Chen, and Z. Dong. 2020. Highlight Every Step: Knowledge Distillation via Collaborative Teaching. IEEE Trans. Cybern. (2020), 1--12.
Recommendations
Compressing Deep Graph Neural Networks via Adversarial Knowledge Distillation
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data MiningDeep graph neural networks (GNNs) have been shown to be expressive for modeling graph-structured data. Nevertheless, the overstacked architecture of deep graph models makes it difficult to deploy and rapidly test on mobile or embedded systems. To ...
Data-free knowledge distillation in neural networks for regression
Highlights- We present data-free knowledge distillation method for regression.
- It adopts ...
AbstractKnowledge distillation has been used successfully to compress a large neural network (teacher) into a smaller neural network (student) by transferring the knowledge of the teacher network with its original training dataset. However, ...
FSKD: Detecting Fake News with Few-Shot Knowledge Distillation
Advanced Data Mining and ApplicationsAbstractThe detection of fake news on social networks is highly desirable and socially beneficial. In real scenarios, there are few labeled news articles and a large number of unlabeled articles. One prominent way is to consider fake news detection as a ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2023
574 pages
ISBN:9798400704147
DOI:10.1145/3625687
- General Chair:
- Rasit Eskicioglu,
- Program Chair:
- Polly Huang,
- Program Co-chair:
- Neal Patwari
Copyright © 2023 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
- SIGARCH: ACM Special Interest Group on Computer Architecture
- SIGBED: ACM Special Interest Group on Embedded Systems
- SIGMETRICS: ACM Special Interest Group on Measurement and Evaluation
- SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
- SIGOPS: ACM Special Interest Group on Operating Systems
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 April 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
Conference
SenSys '23
SenSys '23: 21st ACM Conference on Embedded Networked Sensor Systems
November 12 - 17, 2023
Istanbul, Turkiye
Acceptance Rates
Overall Acceptance Rate 174 of 867 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 123Total Downloads
- Downloads (Last 12 months)123
- Downloads (Last 6 weeks)20
Reflects downloads up to 26 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in