More Web Proxy on the site http://driver.im/

research-article

AXNet: approximate computing using an end-to-end trainable neural network

Authors:

Li JiangAuthors Info & Claims

ICCAD '18: Proceedings of the International Conference on Computer-Aided Design

Article No.: 11, Pages 1 - 8

https://doi.org/10.1145/3240765.3240783

Published: 05 November 2018 Publication History

Abstract

Neural network based approximate computing is a universal architecture promising to gain tremendous energy-efficiency for many error resilient applications. To guarantee the approximation quality, existing works deploy two neural networks (NNs), e.g., an approximator and a predictor. The approximator provides the approximate results, while the predictor predicts whether the input data is safe to approximate with the given quality requirement. However, it is non-trivial and time-consuming to make these two neural network coordinate---they have different optimization objectives---by training them separately. This paper proposes a novel neural network structure---AXNet---to fuse two NNs to a holistic end-to-end trainable NN. Leveraging the philosophy of multi-task learning, AXNet can tremendously improve the invocation (proportion of safe-to-approximate samples) and reduce the approximation error. The training effort also decrease significantly. Experiment results show 50.7% more invocation and substantial cuts of training time when compared to existing neural network based approximate computing framework.

References

[1]

J. Baxter, "Learning internal representations," pp. 311--320, 1995.

Digital Library

[2]

R. Caruana, "Multitask learning," in Learning to learn. Springer, 1998, pp. 95--133.

Digital Library

[3]

R. A. Caruana, "Multitask connectionist learning," in In Proceedings of the 1993 Connectionist Models Summer School. Citeseer, 1993.

[4]

Y. Chen, T. Luo, S. Liu, S. Zhang, L. He, J. Wang, L. Li, T. Chen, Z. Xu, N. Sun et al., "Dadiannao: A machine-learning supercomputer," in Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture. IEEE Computer Society, 2014, pp. 609--622.

Digital Library

[5]

D. Ernst, N. S. Kim, S. Das, S. Pant, R. Rao, T. Pham, C. Ziesler, D. Blaauw, T. Austin, K. Flautner et al., "Razor: A low-power pipeline based on circuit-level timing speculation," in Proceedings of the 36th annual IEEE/ACM International Symposium on Microarchitecture. IEEE Computer Society, 2003, p. 7.

Digital Library

[6]

H. Esmaeilzadeh, A. Sampson, L. Ceze, and D. Burger, "Neural acceleration for general-purpose approximate programs," in Proceedings of the 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture. IEEE Computer Society, 2012, pp. 449--460.

Digital Library

[7]

I. Goodfellow, "Nips 2016 tutorial: Generative adversarial networks," arXiv preprint arXiv:1701.00160, 2016.

[8]

I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning. MIT press Cambridge, 2016, vol. 1.

Digital Library

[9]

S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. Horowitz, and W. J. Dally, "Eie: efficient inference engine on compressed deep neural network," international symposium on computer architecture, vol. 44, no. 3, pp. 243--254, 2016.

Digital Library

[10]

X. He, G. Yan, Y. Han, and X. Li, "Acr: Enabling computation reuse for approximate computing," in Design Automation Conference, 2016, pp. 643--648.

[11]

K. Hornik, "Approximation capabilities of multilayer feedforward networks," Neural networks, vol. 4, no. 2, pp. 251--257, 1991.

Digital Library

[12]

D. S. Khudia, B. Zamirai, M. Samadi, and S. Mahlke, "Rumba: An online quality management system for approximate computing," in Computer Architecture (ISCA), 2015 ACM/IEEE 42nd Annual International Symposium on. IEEE, 2015, pp. 554--566.

Digital Library

[13]

S. Lawrence, I. Burns, A. D. Back, A. C. Tsoi, and C. L. Giles, "Neural network classification and prior class probabilities," neural information processing systems, pp. 299--313, 1998.

Digital Library

[14]

R. Longadge and S. Dongre, "Class imbalance problem in data mining review," arXiv preprint arXiv:1305.1707, 2013.

[15]

D. Mahajan, A. Yazdanbakhsh, J. Park, B. Thwaites, and H. Esmaeilzadeh, "Prediction-based quality control for approximate accelerators," in Second Workshop on Approximate Computing Across the System Stack, WACAS, 2015.

[16]

D. Mahajan, A. Yazdanbakhsh, J. Park, B. Thwaites, and H. Esmaeilzadeh, "Towards statistical guarantees in controlling quality tradeoffs for approximate acceleration," Acm Sigarch Computer Architecture News, vol. 44, no. 3, pp. 66--77, 2016.

Digital Library

[17]

M. Samadi and et al., "Sage: self-tuning approximation for graphics engines," international symposium on microarchitecture, pp. 13--24, 2013.

Digital Library

[18]

X. Sui, A. Lenharth, D. S. Fussell, and K. Pingali, "Proactive control of approximate programs," ACM SIGOPS Operating Systems Review, vol. 50, no. 2, pp. 607--621, 2016.

Digital Library

[19]

T. Wang, Q. Zhang, N. S. Kim, and Q. Xu, "On effective and efficient quality management for approximate computing," in Proceedings of the 2016 International Symposium on Low Power Electronics and Design. ACM, 2016, pp. 156--161.

Digital Library

[20]

C. Xu, X. Wu, W. Yin, Q. Xu, N. Jing, X. Liang, and L. Jiang, "On quality trade-off control for approximate computing using iterative training," in Proceedings of the 54th Annual Design Automation Conference 2017. ACM, 2017, p. 52.

Digital Library

[21]

X. Xu and H. H. Huang, "Exploring data-level error tolerance in high-performance solid-state drives," IEEE Transactions on Reliability, vol. 64, no. 1, pp. 15--30, 2015.

[22]

A. Yazdanbakhsh, D. Mahajan, H. Esmaeilzadeh, and P. Lotfi-Kamran, "Axbench: A multiplatform benchmark suite for approximate computing," IEEE Design & Test, vol. 34, no. 2, pp. 60--68, 2017.

[23]

Z. Zabokrtsky, "Feature engineering in machine learning," Institute of Formal and Applied Linguistics, Charles University in Prague, 2015.

[24]

Q. Zhang and et al., "Approxann: an approximate computing framework for artificial neural network," in Proceedings of the 2015 Design, Automation & Test in Europe Conference & Exhibition. EDA Consortium, 2015, pp. 701--706.

Digital Library

Cited By

Liu FLi HYang NChen YWang ZYang TJiang LKim T(2024)PAAP-HD: PIM-Assisted Approximation for Efficient Hyper-Dimensional ComputingProceedings of the 29th Asia and South Pacific Design Automation Conference10.1109/ASP-DAC58780.2024.10473823(46-51)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1109/ASP-DAC58780.2024.10473823
Jeong YPark JKim RLee S(2024)SEAM: A synergetic energy-efficient approximate multiplier for application demanding substantial computational resourcesIntegration10.1016/j.vlsi.2024.102337(102337)Online publication date: Dec-2024
https://doi.org/10.1016/j.vlsi.2024.102337
Damsgaard HOmetov ANurmi J(2023)Approximation Opportunities in Edge Computing Hardware: A Systematic Literature ReviewACM Computing Surveys10.1145/357277255:12(1-49)Online publication date: 3-Mar-2023
https://dl.acm.org/doi/10.1145/3572772
Show More Cited By

AXNet: approximate computing using an end-to-end trainable neural network
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

A Multitask Learning Neural Network for Short-Term Traffic Speed Prediction and Confidence Estimation
Artificial Neural Networks and Machine Learning – ICANN 2019: Deep Learning
Abstract
To improve predictive accuracy on short-term traffic speed, we proposed a multitask learning neural network (MLNN). MLNN carries out the speed prediction task for three short-terms by the combination of convolution neural network (CNN) and gated ...
GMDH-type neural network algorithm with a feedback loop for structural identification of RBF neural network

In this paper, a Group Method of Data Handling (GMDH)-type neural network algorithm with a feedback loop for structural identification of Radial Basis Function (RBF) neural network is proposed. In case of the GMDH-type neural network, the network ...
Incidence Rate of Hypertension Forecasting Based on Generalized Regression Neural Network
ISCID '10: Proceedings of the 2010 International Symposium on Computational Intelligence and Design - Volume 01

Objective　To study the superiority and application prospect of generalized regression neural network (GRNN) which is used in forecasting the incidence rate of hypertension. Methods　Use meteorological data, including average temperature, average pressure ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICCAD '18: Proceedings of the International Conference on Computer-Aided Design

November 2018

1020 pages

ISBN:9781450359504

DOI:10.1145/3240765

General Chair:
Iris Bahar
Brown Univ. School of Engineering, Providence, RI

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

IEEE-EDS: Electronic Devices Society
IEEE CAS
IEEE CEDA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Shanghai clinical ability construction of The three grade hospital
Shanghai Jiao Tong University Biomedical Engineering Research Foundation
National Natural Science Foundation of China
Shanghai Science and Technology Committee

Conference

ICCAD '18

Sponsor:

IEEE-EDS

ICCAD '18: IEEE/ACM INTERNATIONAL CONFERENCE ON COMPUTER-AIDED DESIGN

November 5 - 8, 2018

California, San Diego

Acceptance Rates

Overall Acceptance Rate 457 of 1,762 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
261
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu FLi HYang NChen YWang ZYang TJiang LKim T(2024)PAAP-HD: PIM-Assisted Approximation for Efficient Hyper-Dimensional ComputingProceedings of the 29th Asia and South Pacific Design Automation Conference10.1109/ASP-DAC58780.2024.10473823(46-51)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1109/ASP-DAC58780.2024.10473823
Jeong YPark JKim RLee S(2024)SEAM: A synergetic energy-efficient approximate multiplier for application demanding substantial computational resourcesIntegration10.1016/j.vlsi.2024.102337(102337)Online publication date: Dec-2024
https://doi.org/10.1016/j.vlsi.2024.102337
Damsgaard HOmetov ANurmi J(2023)Approximation Opportunities in Edge Computing Hardware: A Systematic Literature ReviewACM Computing Surveys10.1145/357277255:12(1-49)Online publication date: 3-Mar-2023
https://dl.acm.org/doi/10.1145/3572772
Raza MJaved SKazmi MAziz AUl Haque MQazi S(2023)Approximate Computing: Hardware and Software Techniques, Tools and Their ApplicationsJournal of Circuits, Systems and Computers10.1142/S021812662430001033:04Online publication date: 20-Sep-2023
https://doi.org/10.1142/S0218126624300010
Menon HDiffenderfer JGeorgakoudis GLaguna ILam MOsei-Kuffuor DParasyris KVanover J(2023)Approximate High-Performance Computing: A Fast and Energy-Efficient Computing Paradigm in the Post-Moore EraIT Professional10.1109/MITP.2023.325464225:2(7-15)Online publication date: Mar-2023
https://doi.org/10.1109/MITP.2023.3254642
Sathidevi LSharma AWu NJiao XHao C(2023)PreAxC: Error Distribution Prediction for Approximate Computing Quality Control using Graph Neural Networks2023 24th International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED57927.2023.10129393(1-7)Online publication date: 5-Apr-2023
https://doi.org/10.1109/ISQED57927.2023.10129393
Saravanan RBavikadi SRai SKumar APudukotai Dinakarrao S(2023)Reconfigurable FET Approximate Computing-based Accelerator for Deep Learning Applications2023 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS46773.2023.10181758(1-5)Online publication date: 21-May-2023
https://doi.org/10.1109/ISCAS46773.2023.10181758
Barua HMondal K(2023)Cloud Big Data Mining and Analytics: Bringing Greenness and Acceleration in the CloudMachine Learning for Data Science Handbook10.1007/978-3-031-24628-9_22(491-510)Online publication date: 26-Feb-2023
https://doi.org/10.1007/978-3-031-24628-9_22
Bavikadi SDhavlle AGanguly AHaridass AHendy HMerkel CReddi VSutradhar PJoseph APudukotai Dinakarrao S(2022)A Survey on Machine Learning Accelerators and Evolutionary Hardware PlatformsIEEE Design & Test10.1109/MDAT.2022.316112639:3(91-116)Online publication date: Jun-2022
https://doi.org/10.1109/MDAT.2022.3161126
Saeedi SCarpegna ASavino ADi Carlo S(2022)Prediction of the Impact of Approximate Computing on Spiking Neural Networks via Interval Arithmetic2022 IEEE 23rd Latin American Test Symposium (LATS)10.1109/LATS57337.2022.9936999(1-6)Online publication date: 5-Sep-2022
https://doi.org/10.1109/LATS57337.2022.9936999
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents