More Web Proxy on the site http://driver.im/

research-article

Deep-DFR: A Memristive Deep Delayed Feedback Reservoir Computing System with Hybrid Neural Network Topology

Authors:

Yang YiAuthors Info & Claims

DAC '19: Proceedings of the 56th Annual Design Automation Conference 2019

Article No.: 54, Pages 1 - 6

https://doi.org/10.1145/3316781.3317796

Published: 02 June 2019 Publication History

Abstract

Deep neural networks (DNNs), the brain-like machine learning architecture, have gained immense success in data-extensive applications. In this work, a hybrid structured deep delayed feedback reservoir (Deep-DFR) computing model is proposed and fabricated. Our Deep-DFR employs memristive synapses working in a hierarchical information processing fashion with DFR modules as the readout layer, leading our proposed deep learning structure to be both depth-in-space and depth-in-time. Our fabricated prototype along with experimental results demonstrate its high energy efficiency with low hardware implementation cost. With applications on the image classification, MNIST and SVHN, our Deep-DFR yields a 1.26~7.69X reduction on the testing error compared to state-of-the-art DNN designs.

References

[1]

LeCun, Y., Y. Bengio, and G. Hinton, Deep learning. nature, 2015. 521(7553): p. 436.

[2]

Jaeger, H., The "echo state" approach to analysing and training recurrent neural networks-with an erratum note. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report, 2001. 148(34): p. 13.

[3]

Pascanu, R., et al., How to construct deep recurrent neural networks. arXiv preprint arXiv:1312.6026, 2013.

[4]

Appeltant, L., et al., Information processing using a single dynamical node as complex system. Nature communications, 2011. 2: p. 468.

[5]

Ma, Q., L. Shen, and G.W. Cottrell, Deep-ESN: A Multiple Projection-encoding Hierarchical Reservoir Computing Framework. arXiv preprint arXiv:1711.05255, 2017.

[6]

Chiu, P.-F., et al., Low store energy, low vddmin, 8t2r nonvolatile latch and sram with vertical-stacked resistive memory (memristor) devices for low power mobile applications. IEEE Journal of Solid-State Circuits, 2012. 47(6): p. 1483--1496.

[7]

Hagan, M.T., et al., Neural network design. Vol. 20. 1996: Pws Pub. Boston.

Digital Library

[8]

Bai, K., et al. Enabling An New Era of Brain-inspired Computing: Energy-efficient Spiking Neural Network with Ring Topology. in 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC). 2018. IEEE.

Digital Library

[9]

Radford, A., L. Metz, and S. Chintala, Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.

[10]

Wu, J., Symmetric functional differential equations and neural networks with memory. Transactions of the American mathematical society, 1998. 350(12): p. 4799--4838.

[11]

Beuter, A., J. Bélair, and C. Labrie, Feedback and delays in neurological diseases: A modeling study using gynamical systems. Bulletin of mathematical biology, 1993. 55(3): p. 525--541.

[12]

Mackey, M.C. and L. Glass, Oscillation and chaos in physiological control systems. Science, 1977. 197(4300): p. 287--289.

[13]

Ikeda, K. and K. Matsumoto, High-dimensional chaotic behavior in systems with time-delayed feedback. Physica D: Nonlinear Phenomena, 1987. 29(1-2): p. 223--235.

Digital Library

[14]

Boedecker, J., et al., Information processing in echo state networks at the edge of chaos. Theory in Biosciences, 2012. 131(3): p. 205--213.

[15]

Zhao, C., et al., Interspike-Interval-Based Analog Spike-Time-Dependent Encoder for Neuromorphic Processors. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2017. 25(8): p. 2193--2205.

Digital Library

[16]

Bai, K. and Y. Yi, DFR: An Energy-efficient Analog Delay Feedback Reservoir Computing System for Brain-inspired Computing. ACM Journal on Emerging Technologies in Computing Systems (JETC), 2018. 14(4): p. 45.

Digital Library

[17]

Molter, T.W. and M.A. Nugent. The generalized metastable switch memristor model. in CNNA 2016; 15th International Workshop on Cellular Nanoscale Networks and their Applications; Proceedings of. 2016. VDE.

[18]

LeCun, Y., C. Cortes, and C. Burges, Mnist handwritten digit database. AT&T Labs. 2010.

[19]

Netzer, Y., et al., Reading digits in natural images with unsupervised feature learning. 2011.

[20]

Whatmough, P.N., et al. 14.3 A 28nm SoC with a 1.2 GHz 568nJ/prediction sparse deep-neural-network engine with> 0.1 timing error rate tolerance for IoT applications. in 2017 IEEE International Solid-State Circuits Conference (ISSCC). 2017. IEEE.

[21]

Yu, S., et al. Binary neural network with 16 Mb RRAM macro chip for classification and online training. in 2016 IEEE International Electron Devices Meeting (IEDM). 2016. IEEE.

[22]

Mozafari, M., et al., Combining stdp and reward-modulated stdp in deep convolutional spiking neural networks for digit recognition. arXiv preprint arXiv:1804.00227, 2018.

[23]

Umuroglu, Y., et al. Finn: A framework for fast, scalable binarized neural network inference. in Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2017. ACM.

Digital Library

[24]

Jerry, M., et al. Ferroelectric FET analog synapse for acceleration of deep neural network training. in 2017 IEEE International Electron Devices Meeting (IEDM). 2017. IEEE.

Cited By

Sedki KYi Y(2024)Advancing Analog Reservoir Computing through Temporal Attention and MLP Integration2024 25th International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED60706.2024.10528762(1-8)Online publication date: 3-Apr-2024
https://doi.org/10.1109/ISQED60706.2024.10528762
Wu XLin ZDeng JLi JFeng Y(2024)Nonmasking-based reservoir computing with a single dynamic memristor for image recognitionNonlinear Dynamics10.1007/s11071-024-09338-9112:8(6663-6678)Online publication date: 6-Mar-2024
https://doi.org/10.1007/s11071-024-09338-9
Zins NZhang YAn H(2023)Implementation of Associative Memory Learning in Mobile Robots Using Neuromorphic ComputingNeuromorphic Computing10.5772/intechopen.110364Online publication date: 15-Nov-2023
https://doi.org/10.5772/intechopen.110364
Show More Cited By

Deep-DFR: A Memristive Deep Delayed Feedback Reservoir Computing System with Hybrid Neural Network Topology
1. Computer systems organization
  1. Architectures
    1. Other architectures
2. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals
Abstract
In the Motor Imagery (MI)-based Brain Computer Interface (BCI), users' intention is converted into a control signal through processing a specific pattern in brain signals reflecting motor characteristics. There are such restrictions as ...
Highlights
- Data Augmentation by the cropping method to increase the number of trials.
- Time-...
A dyadic multi-resolution deep convolutional neural wavelet network for image classification

For almost the past four decades, image classification has gained a lot of attention in the field of pattern recognition due to its application in various fields. Given its importance, several approaches have been proposed up to now. In this paper, we ...
Richer feature for image classification with super and sub kernels based on deep convolutional neural network

The concepts of super convolutional kernel and sub convolutional kernel are proposed inspired by the process that human observes objects.A novel Parallel Crossing Deep Convolutional Neural Network (PCDCNN) model is designed to simulate the human ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DAC '19: Proceedings of the 56th Annual Design Automation Conference 2019

June 2019

1378 pages

ISBN:9781450367257

DOI:10.1145/3316781

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGDA: ACM Special Interest Group on Design Automation
IEEE-CEDA

In-Cooperation

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

air force research lab

Conference

DAC '19

Sponsor:

SIGDA

DAC '19: The 56th Annual Design Automation Conference 2019

June 2 - 6, 2019

NV, Las Vegas, USA

Acceptance Rates

Overall Acceptance Rate 1,770 of 5,499 submissions, 32%

Upcoming Conference

DAC '25

Sponsor:
sigda

62nd ACM/IEEE Design Automation Conference

June 22 - 26, 2025

San Francisco , CA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
278
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)2

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sedki KYi Y(2024)Advancing Analog Reservoir Computing through Temporal Attention and MLP Integration2024 25th International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED60706.2024.10528762(1-8)Online publication date: 3-Apr-2024
https://doi.org/10.1109/ISQED60706.2024.10528762
Wu XLin ZDeng JLi JFeng Y(2024)Nonmasking-based reservoir computing with a single dynamic memristor for image recognitionNonlinear Dynamics10.1007/s11071-024-09338-9112:8(6663-6678)Online publication date: 6-Mar-2024
https://doi.org/10.1007/s11071-024-09338-9
Zins NZhang YAn H(2023)Implementation of Associative Memory Learning in Mobile Robots Using Neuromorphic ComputingNeuromorphic Computing10.5772/intechopen.110364Online publication date: 15-Nov-2023
https://doi.org/10.5772/intechopen.110364
Bai KThiem CLombardi JLiang YYi Y(2023)Design Strategies and Applications of Reservoir Computing: Recent Trends and Prospects [Feature]IEEE Circuits and Systems Magazine10.1109/MCAS.2023.332549623:4(10-33)Online publication date: Dec-2024
https://doi.org/10.1109/MCAS.2023.3325496
Gan VLiang YLi LLiu LYi Y(2021)A Cost-Efficient Digital ESN Architecture on FPGA for OFDM Symbol DetectionACM Journal on Emerging Technologies in Computing Systems10.1145/344001717:4(1-15)Online publication date: 30-Jun-2021
https://dl.acm.org/doi/10.1145/3440017
An HAl-Mamun MOrlowski MLiu LYi Y(2021)Robust Deep Reservoir Computing Through Reliable Memristor With Improved Heat Dissipation CapabilityIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2020.300253940:3(574-583)Online publication date: Mar-2021
https://doi.org/10.1109/TCAD.2020.3002539
Liu SYi Y(2020)Quantized Neural Networks and Neuromorphic Computing for Embedded SystemsIntelligent System and Computing [Working Title]10.5772/intechopen.91835Online publication date: 30-Mar-2020
https://doi.org/10.5772/intechopen.91835
An HHa DYi YBentley WPayne GLoscri V(2020)Powering next-generation industry 4.0 by a self-learning and low-power neuromorphic systemProceedings of the 7th ACM International Conference on Nanoscale Computing and Communication10.1145/3411295.3411302(1-6)Online publication date: 23-Sep-2020
https://dl.acm.org/doi/10.1145/3411295.3411302
Bai KAn QLiu LYi Y(2020)A Training-Efficient Hybrid-Structured Deep Neural Network With Reconfigurable Memristive SynapsesIEEE Transactions on Very Large Scale Integration (VLSI) Systems10.1109/TVLSI.2019.294226728:1(62-75)Online publication date: Jan-2020
https://doi.org/10.1109/TVLSI.2019.2942267
Nowshin FLiu LYi Y(2020)Energy Efficient and Adaptive Analog IC Design for Delay-Based Reservoir Computing2020 IEEE 63rd International Midwest Symposium on Circuits and Systems (MWSCAS)10.1109/MWSCAS48704.2020.9184677(592-595)Online publication date: Aug-2020
https://doi.org/10.1109/MWSCAS48704.2020.9184677
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten