More Web Proxy on the site http://driver.im/

research-article

Semi-Trained Memristive Crossbar Computing Engine with In Situ Learning Accelerator

Authors:

Abdullah M. Zyarah,

Dhireesha KudithipudiAuthors Info & Claims

ACM Journal on Emerging Technologies in Computing Systems (JETC), Volume 14, Issue 4

Article No.: 43, Pages 1 - 16

https://doi.org/10.1145/3233987

Published: 27 November 2018 Publication History

Abstract

On-device intelligence is gaining significant attention recently as it offers local data processing and low power consumption. In this research, an on-device training circuitry for threshold-current memristors integrated in a crossbar structure is proposed. Furthermore, alternate approaches of mapping the synaptic weights into fully trained and semi-trained crossbars are investigated. In a semi-trained crossbar, a confined subset of memristors are tuned and the remaining subset of memristors are not programmed. This translates to optimal resource utilization and power consumption, compared to a fully programmed crossbar. The semi-trained crossbar architecture is applicable to a broad class of neural networks. System level verification is performed with an extreme learning machine for binomial and multinomial classification. The total power for a single 4 × 4 layer network, when implemented in IBM 65nm node, is estimated to be ≈42.16μW and the area is estimated to be 26.48μm × 22.35μm.

References

[1]

Fabien Alibart, Elham Zamanidoost, and Dmitri B. Strukov. 2013. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nature Commun. 4 (June 2013), 2072.

[2]

Joshua E. Auerbach, Chrisantha Fernando, and Dario Floreano. 2014. Online extreme evolutionary learning machines. In Proceedings of the 14th International Conference on the Synthesis and Simulation of Living Systems: Artificial Life 14. The MIT Press, 465--472.

[3]

Julien Borghetti, Gregory S. Snider, Philip J. Kuekes, J. Joshua Yang, Duncan R. Stewart, and R. Stanley Williams. 2010. Memristive switches enable stateful logic operations via material implication. Nature 464, 7290 (2010), 873--876.

[4]

Gangotree Chakma, Md. Musabbir Adnan, Austin R. Wyer, Ryan Weiss, Catherine D. Schuman, and Garrett S. Rose. 2018. Memristive mixed-signal neuromorphic systems: Energy-efficient learning at the circuit-level. IEEE J. Emerg. Select. Topics Circ. Syst. 8, 1 (2018), 125--136.

[5]

Leon Chua. 1971. Memristor-the missing circuit element. IEEE Trans. Circ. Theory 18, 5 (1971), 507--519.

[6]

Deliang Fan, Mrigank Sharad, and Kaushik Roy. 2014. Design and synthesis of ultralow energy spin-memristor threshold logic. IEEE Trans. Nanotechnol. 13, 3 (2014), 574--583.

Digital Library

[7]

Md Raqibul Hasan. 2016. Memristor Based Low Power High Throughput Circuits and Systems Design. Ph.D. Dissertation. University of Dayton.

[8]

Raqibul Hasan, Tarek M. Taha, and Chris Yakopcic. 2017. On-chip training of memristor crossbar-based multi-layer neural networks. Microelectron. J. 66 (2017), 31--40.

Digital Library

[9]

Miao Hu, Hai Li, Yiran Chen, Qing Wu, Garrett S. Rose, and Richard W. Linderman. 2014. Memristor crossbar-based neuromorphic computing system: A case study. IEEE Trans. Neural Netw. Learn. Syst. 25, 10 (2014), 1864--1878.

[10]

Guang-Bin Huang. 2014. An insight into extreme learning machines: Random neurons, random features and kernels. Cogn. Comput. 6, 3 (2014), 376--390.

[11]

Guang-Bin Huang, Hongming Zhou, Xiaojian Ding, and Rui Zhang. 2012. Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst., Man, Cybernet., Part B (Cybernet.) 42, 2 (2012), 513--529.

Digital Library

[12]

Guang-Bin Huang, Qin-Yu Zhu, and Chee-Kheong Siew. 2004. Extreme learning machine: A new learning scheme of feedforward neural networks. In Proceedings of the IEEE International Joint Conference on Neural Networks, Vol. 2. IEEE, 985--990.

[13]

Guang-Bin Huang, Qin-Yu Zhu, and Chee-Kheong Siew. 2006. Extreme learning machine: Theory and applications. Neurocomputing 70, 1 (2006), 489--501.

[14]

Giacomo Indiveri and Shih-Chii Liu. 2015. Memory and information processing in neuromorphic systems. Proc. IEEE 103, 8 (2015), 1379--1397.

[15]

Robert A. Jacobs. 1988. Increased rates of convergence through learning rate adaptation. Neural Netw. 1, 4 (1988), 295--307.

[16]

Sung Hyun Jo, Ting Chang, Idongesit Ebong, Bhavitavya B. Bhadviya, Pinaki Mazumder, and Wei Lu. 2010. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 4 (2010), 1297--1301.

[17]

Liyanaarachchi Lekamalage Chamara Kasun, Hongming Zhou, Guang-Bin Huang, and Chi Man Vong. 2013. Representational learning with ELMs for big data. IEEE Intell. Syst. 28, 6 (2013), 31--34.

[18]

Takayuki Kawahara, Riichiro Takemura, Katsuya Miura, Jun Hayakawa, Shoji Ikeda, Young Min Lee, Ryutaro Sasaki, Yasushi Goto, Kenchi Ito, Toshiyasu Meguro, et al. 2008. 2 Mb SPRAM (spin-transfer torque RAM) with bit-by-bit bi-directional current write and parallelizing-direction current read. IEEE J. Solid-State Circ. 43, 1 (2008), 109--120.

[19]

Hyongsuk Kim, Maheshwar Pd. Sah, Changju Yang, Tamás Roska, and Leon O. Chua. 2012. Neural synaptic weighting with a pulse-based memristor circuit. IEEE Trans. Circ. Syst. I: Reg. Papers 59, 1 (2012), 148--158.

[20]

Shahar Kvatinsky, Eby G. Friedman, Avinoam Kolodny, and Uri C. Weiser. 2013. TEAM: Threshold adaptive memristor model. IEEE Trans. Circ. Syst. I: Reg. Papers 60, 1 (2013), 211--221.

[21]

Yann LeCun. 1998. The MNIST database of handwritten digits. Retrieved from http://yann.lecun.com/exdb/mnist/.

[22]

M. Lichman. 2013. UCI Machine Learning Repository. Retrieved from http://archive.ics.uci.edu/ml.

[23]

Roberto Marani, Gennaro Gelao, and Anna Gina Perri. 2015. A review on memristor applications. arXiv Preprint arXiv:1506.06899 (2015).

[24]

Cory Merkel. 2017. Current-mode memristor crossbars for neuromemristive systems. arXiv Preprint arXiv:1707.05316 (2017).

[25]

Cory Merkel and Dhireesha Kudithipudi. 2014. Neuromemristive extreme learning machines for pattern classification. In Proceedings of the IEEE Computer Society Annual Symposium on VLSI (ISVLSI’14). IEEE, 77--82.

Digital Library

[26]

Yoh-Han Pao, Gwang-Hoon Park, and Dejan J. Sobajic. 1994. Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6, 2 (1994), 163--180.

[27]

André Bannwart Perina, Paulo Matias, Eduardo Marques, Vanderlei Bonato, João Miguel Gago Pontes De Brito, et al. 2017. Exploiting kant and kimura matrix inversion algorithm on FPGA. In Proceedings of the Euromicro Conference on Digital System Design (DSD’17). IEEE, 516--519.

[28]

Mirko Prezioso, Farnood Merrikh-Bayat, B. D. Hoskins, G. C. Adam, Konstantin K. Likharev, and Dmitri B. Strukov. 2015. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 7550 (2015), 61--64.

[29]

Maheshwar Pd. Sah, Changju Yang, Hyongsuk Kim, and Leon O. Chua. 2012. Memristor circuit for artificial synaptic weighting of pulse inputs. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS’12). IEEE, 1604--1607.

[30]

Greg S. Snider. 2008. Spike-timing-dependent learning in memristive nanodevices. In Proceedings of the IEEE International Symposium on Nanoscale Architectures (NANOARCH’08). IEEE, 85--92.

Digital Library

[31]

Daniel Soudry, Dotan Di Castro, Asaf Gal, Avinoam Kolodny, and Shahar Kvatinsky. 2015. Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans. Neural Netw. Learn. Syst. 26, 10 (2015), 2408--2421.

[32]

Dmitri B. Strukov, Gregory S. Snider, Duncan R. Stewart, and R. Stanley Williams. 2008. The missing memristor found. Nature 453, 7191 (2008), 80--83.

[33]

Manan Suri, Vivek Parmar, Gilbert Sassine, and Fabien Alibart. 2015. OXRAM based ELM architecture for multi-class classification applications. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’15). IEEE, 1--8.

[34]

Tarek M. Taha, Raqibul Hasan, and Chris Yakopcic. 2014. Memristor crossbar-based multicore neuromorphic processors. In Proceedings of the 27th IEEE International System-on-Chip Conference (SOCC’14). IEEE, 383--389.

[35]

Enyi Yao and Arindam Basu. 2017. VLSI extreme learning machine: A design space exploration. IEEE Trans. Very Large Scale Integr. Syst. 25, 1 (2017), 60--74.

Digital Library

[36]

Abdullah M. Zyarah and Dhireesha Kudithipudi. 2017. Extreme learning machine as a generalizable classification engine. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’17). IEEE, 3371--3376.

[37]

Abdullah M. Zyarah, Nicholas Soures, Lydia Hays, Robin B. Jacobs-Gedrim, Sapan Agarwal, Matthew Marinella, and Dhireesha Kudithipudi. 2017. Ziksa: On-chip learning accelerator with memristor crossbars for multilevel neural networks. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS’17). IEEE, 1--4.

Cited By

Zyarah AKudithipudi D(2024)Time-Series Forecasting and Sequence Learning Using Memristor-based Reservoir SystemACM Transactions on Embedded Computing Systems10.1145/370344624:1(1-17)Online publication date: 5-Nov-2024
https://dl.acm.org/doi/10.1145/3703446
Zohora FKaria VSoures NKudithipudi D(2024)Probabilistic metaplasticity for continual learning with memristors in spiking networksScientific Reports10.1038/s41598-024-78290-w14:1Online publication date: 27-Nov-2024
https://doi.org/10.1038/s41598-024-78290-w
Titirsha TSong SDas AKrichmar JDutt NKandasamy NCatthoor F(2022)Endurance-Aware Mapping of Spiking Neural Networks to Neuromorphic HardwareIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.306559133:2(288-301)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TPDS.2021.3065591
Show More Cited By

Index Terms

Semi-Trained Memristive Crossbar Computing Engine with In Situ Learning Accelerator
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Hardware
  1. Emerging technologies

Recommendations

Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars
ISLPED '22: Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and Design

Spiking Neural Networks (SNNs) have recently emerged as the low-power alternative to Artificial Neural Networks (ANNs) owing to their asynchronous, sparse, and binary information processing. To improve the energy-efficiency and throughput, SNNs can be ...
Memristive Crossbar Mapping for Neuromorphic Computing Systems on 3D IC

In recent years, neuromorphic computing systems based on memristive crossbar have provided a promising solution to enable acceleration of neural networks. However, most of the neural networks used in realistic applications are often sparse. If such ...
Comparison of two memristor based neural network learning schemes for crossbar architecture
IWANN'13: Proceedings of the 12th international conference on Artificial Neural Networks: advances in computational intelligence - Volume Part I

This paper compares two neural network learning schemes in crossbar architecture, using memristive elements. Novel memristive crossbar architecture with dense synaptic connections suitable for online training was developed. Training algorithms and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Journal on Emerging Technologies in Computing Systems

ACM Journal on Emerging Technologies in Computing Systems Volume 14, Issue 4

Special Issue on Neuromorphic Computing

October 2018

164 pages

ISSN:1550-4832

EISSN:1550-4840

DOI:10.1145/3294068

Editor:
Yuan Xie
University of California, Santa Barbara, USA

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 27 November 2018

Accepted: 01 June 2018

Revised: 01 May 2018

Received: 01 December 2017

Published in JETC Volume 14, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

AirForce Research Laboratory

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
180
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zyarah AKudithipudi D(2024)Time-Series Forecasting and Sequence Learning Using Memristor-based Reservoir SystemACM Transactions on Embedded Computing Systems10.1145/370344624:1(1-17)Online publication date: 5-Nov-2024
https://dl.acm.org/doi/10.1145/3703446
Zohora FKaria VSoures NKudithipudi D(2024)Probabilistic metaplasticity for continual learning with memristors in spiking networksScientific Reports10.1038/s41598-024-78290-w14:1Online publication date: 27-Nov-2024
https://doi.org/10.1038/s41598-024-78290-w
Titirsha TSong SDas AKrichmar JDutt NKandasamy NCatthoor F(2022)Endurance-Aware Mapping of Spiking Neural Networks to Neuromorphic HardwareIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.306559133:2(288-301)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TPDS.2021.3065591
Jot SZyarah AKurinec SNi KZohora FKudithipudi D(2020)FeFET-Based Neuromorphic Architecture with On-Device Feedback Alignment Training2020 21st International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED48828.2020.9137035(317-322)Online publication date: Mar-2020
https://doi.org/10.1109/ISQED48828.2020.9137035
Zyarah AKudithipudi D(2019)Neuromemristive Multi-Layer Random Projection Network with On-Device Learning2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8851735(1-8)Online publication date: Jul-2019
https://doi.org/10.1109/IJCNN.2019.8851735

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents