More Web Proxy on the site http://driver.im/

research-article

Spin Orbit Torque Device based Stochastic Multi-bit Synapses for On-chip STDP Learning

Authors:

Jongsun ParkAuthors Info & Claims

ISLPED '18: Proceedings of the International Symposium on Low Power Electronics and Design

Article No.: 21, Pages 1 - 6

https://doi.org/10.1145/3218603.3218654

Published: 23 July 2018 Publication History

Abstract

As a large number of neurons and synapses are needed in spike neural network (SNN) design, emerging devices have been employed to implement synapses and neurons. In this paper, we present a stochastic multi-bit spin orbit torque (SOT) memory based synapse, where only one SOT device is switched for potentiation and depression using modified Gray code. The modified Gray code based approach needs only N devices to represent 2N levels of synapse weights. Early read termination scheme is also adopted to reduce the power consumption of training process by turning off less associated neurons and its ADCs. For MNIST dataset, with comparable classification accuracy, the proposed SNN architecture using 3-bit synapse achieves 68.7% reduction of ADC overhead compared to the conventional 8-level synapse.

References

[1]

J. Schmidhuber, "Deep learning in neural networks: An overview," Neural Netw., vol. 61, pp. 85--117, Jan. 2015.

Digital Library

[2]

P. U. Diehl and M. Cook, "Unsupervised learning of digit recognition using spike-timing-dependent plasticity," Front. Comput. Neurosci., vol. 9, 2015.

[3]

Q. Wang, Y. Li, B. Shao, S. Dey, and P. Li, "Energy Efficient Parallel Neuromorphic Architectures with Approximate Arithmetic on FPGA," Neurocomput., vol. 221, no. C, pp. 146--158, Jan. 2017.

Digital Library

[4]

J. s Seo et al., "A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons," in 2011 IEEE Custom Integrated Circuits Conference (CICC), 2011, pp. 1--4.

[5]

Q. Wang, Y. Kim, and P. Li, "Neuromorphic Processors with Memristive Synapses: Synaptic Interface and Architectural Exploration," J Emerg Technol Comput Syst, vol. 12, no. 4, p. 35:1--35:22, May 2016.

Digital Library

[6]

A. F. Vincent et al., "Spin-Transfer Torque Magnetic Memory as a Stochastic Memristive Synapse for Neuromorphic Systems," IEEE Trans. Biomed. Circuits Syst., vol. 9, no. 2, pp. 166--174, Apr. 2015.

[7]

G. Srinivasan, A. Sengupta, and K. Roy, "Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning," Sci. Rep., vol. 6, p. srep29545, Jul. 2016.

[8]

A. Sengupta, A. Banerjee, and K. Roy, "Hybrid Spintronic-CMOS Spiking Neural Network with On-Chip Learning: Devices, Circuits, and Systems," Phys. Rev. Appl., vol. 6, no. 6, p. 064003, Dec. 2016.

[9]

J. Bill and R. Legenstein, "A compound memristive synapse model for statistical learning through STDP in spiking neural networks," Front. Neurosci., vol. 8, 2014.

[10]

D. Zhang, L. Zeng, Y. Zhang, W. Zhao, and J. O. Klein, "Stochastic spintronic device based synapses and spiking neurons for neuromorphic computation," in 2016 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH), 2016, pp. 173--178.

[11]

N. Burkitt, "A Review of the Integrate-and-fire Neuron Model: I. Homogeneous Synaptic Input," Biol Cybern, vol. 95, no. 1, pp. 1--19, Jun. 2006.

Digital Library

[12]

B. Rajendran and F. Alibart, "Neuromorphic Computing Based on Emerging Memory Technologies," IEEE J. Emerg. Sel. Top. Circuits Syst., vol. 6, no. 2, pp. 198--211, Jun. 2016.

[13]

L. Liu, C.-F. Pai, Y. Li, H. W. Tseng, D. C. Ralph, and R. A. Buhrman, "Spin-Torque Switching with the Giant Spin Hall Effect of Tantalum," Science, vol. 336, no. 6081, pp. 555--558, May 2012.

[14]

K.-S. Lee, S.-W. Lee, B.-C. Min, and K.-J. Lee, "Thermally activated switching of perpendicular magnet by spin-orbit spin torque," Appl. Phys. Lett., vol. 104, no. 7, p. 072413, Feb. 2014.

[15]

T. Branco and K. Staras, "The probability of neurotransmitter release: variability and feedback control at single synapses," Nat. Rev. Neurosci., vol. 10, no. 5, pp. 373--383, 2009.

[16]

B. B. Averbeck, P. E. Latham, and A. Pouget, "Neural correlations, population coding and computation," Nat. Rev. Neurosci., vol. 7, no. 5, pp. 358--366, May 2006.

[17]

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278--2324, Nov. 1998.

[18]

Z. Huang and P. Zhong, "An architectural power estimator for analog-to-digital converters," in IEEE International Conference on Computer Design: VLSI in Computers and Processors, 2004. ICCD 2004. Proceedings., 2004, pp. 397--400.

Digital Library

[19]

J. Kim, A. Chen, B. Behin-Aein, S. Kumar, J. P. Wang, and C. H. Kim, "A technology-agnostic MTJ SPICE model with user-defined dimensions for STT-MRAM scalability studies," in 2015 IEEE Custom Integrated Circuits Conference (CICC), 2015, pp. 1--4.

Cited By

Verma GNisar ADhull SKaushik B(2023)Neuromorphic Accelerator for Spiking Neural Network Using SOT-MRAM Crossbar ArrayIEEE Transactions on Electron Devices10.1109/TED.2023.331735770:11(6012-6020)Online publication date: Nov-2023
https://doi.org/10.1109/TED.2023.3317357
Kim TJang YKang MPark BLee KPark J(2022)SOT-MRAM Digital PIM Architecture With Extended Parallelism in Matrix MultiplicationIEEE Transactions on Computers10.1109/TC.2022.315527771:11(2816-2828)Online publication date: 1-Nov-2022
https://doi.org/10.1109/TC.2022.3155277
Kwak MKim Y(2022)Do Not Forget: Exploiting Stability-Plasticity Dilemma to Expedite Unsupervised SNN Training for Neuromorphic Processors2022 IEEE 40th International Conference on Computer Design (ICCD)10.1109/ICCD56317.2022.00069(419-426)Online publication date: Oct-2022
https://doi.org/10.1109/ICCD56317.2022.00069
Show More Cited By

Index Terms

Spin Orbit Torque Device based Stochastic Multi-bit Synapses for On-chip STDP Learning
1. Hardware
  1. Emerging technologies
    1. Analysis and design of emerging devices and systems
      1. Emerging architectures
    2. Spintronics and magnetic technologies
  2. Very large scale integration design
    1. Application-specific VLSI designs
      1. Application specific integrated circuits

Recommendations

GABA inhibition modulates NMDA-R mediated spike timing dependent plasticity (STDP) in a biophysical model

Spike timing dependent plasticity (STDP) has been demonstrated in various neural systems of many animals. It has been shown that STDP depends on the target and the location of the synapse and is dynamically regulated by the activity of adjacent synapses,...
A calcium-influx-dependent plasticity model exhibiting multiple STDP curves
Abstract
Hebbian plasticity means that if the firing of two neurons is correlated, then their connection is strengthened. Conversely, uncorrelated firing causes a decrease in synaptic strength. Spike-timing-dependent plasticity (STDP) represents one ...
How Bursts Shape the STDP Curve in the Presence/Absence of GABAergic Inhibition
ICANN '09: Proceedings of the 19th International Conference on Artificial Neural Networks: Part I

It has been known for some time that the synapses of the CA1 pyramidal cells are surprisingly unreliable at signalling the arrival of single spikes to the postsynaptic neuron [2]. On the other hand, bursts of spikes are reliably signalled, because ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISLPED '18: Proceedings of the International Symposium on Low Power Electronics and Design

July 2018

327 pages

ISBN:9781450357043

DOI:10.1145/3218603

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

ISLPED '18

Sponsor:

SIGDA

ISLPED '18: International Symposium on Low Power Electronics and Design

July 23 - 25, 2018

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 398 of 1,159 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
396
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)1

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Verma GNisar ADhull SKaushik B(2023)Neuromorphic Accelerator for Spiking Neural Network Using SOT-MRAM Crossbar ArrayIEEE Transactions on Electron Devices10.1109/TED.2023.331735770:11(6012-6020)Online publication date: Nov-2023
https://doi.org/10.1109/TED.2023.3317357
Kim TJang YKang MPark BLee KPark J(2022)SOT-MRAM Digital PIM Architecture With Extended Parallelism in Matrix MultiplicationIEEE Transactions on Computers10.1109/TC.2022.315527771:11(2816-2828)Online publication date: 1-Nov-2022
https://doi.org/10.1109/TC.2022.3155277
Kwak MKim Y(2022)Do Not Forget: Exploiting Stability-Plasticity Dilemma to Expedite Unsupervised SNN Training for Neuromorphic Processors2022 IEEE 40th International Conference on Computer Design (ICCD)10.1109/ICCD56317.2022.00069(419-426)Online publication date: Oct-2022
https://doi.org/10.1109/ICCD56317.2022.00069
Jang YKang GKim TSeo YLee KPark BPark J(2021)Stochastic SOT device based SNN architecture for On-chip Unsupervised STDP LearningIEEE Transactions on Computers10.1109/TC.2021.3119180(1-1)Online publication date: 2021
https://doi.org/10.1109/TC.2021.3119180
Verma GBindal NNisar ADhull SKaushik B(2021)Advances in Neuromorphic Spin-Based Spiking Neural Networks: A reviewIEEE Nanotechnology Magazine10.1109/MNANO.2021.309821915:5(33-44)Online publication date: Oct-2021
https://doi.org/10.1109/MNANO.2021.3098219
Shreya SGuglani SKaushik BRoy S(2020)Statistical Analysis of Temperature Variability on the Write Efficiency of Spin-Orbit Torque MRAM using Polynomial Chaos Metamodels2020 21st International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED48828.2020.9137004(87-92)Online publication date: Mar-2020
https://doi.org/10.1109/ISQED48828.2020.9137004

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents