More Web Proxy on the site http://driver.im/

research-article

Mapping Very Large Scale Spiking Neuron Network to Neuromorphic Hardware

Authors:

Shuiguang Deng,

Gang PanAuthors Info & Claims

ASPLOS 2023: Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3

Pages 419 - 432

https://doi.org/10.1145/3582016.3582038

Published: 25 March 2023 Publication History

Abstract

Neuromorphic hardware is a multi-core computer system specifically designed to run Spiking Neuron Network (SNN) applications. As the scale of neuromorphic hardware increases, it becomes very challenging to efficiently map a large SNN to hardware. In this paper, we proposed an efficient approach to map very large scale SNN applications to neuromorphic hardware, aiming to reduce energy consumption, spike latency, and on-chip network communication congestion. The approach consists of two steps. Firstly, it solves the initial placement using the Hilbert curve, a space-filling curve with unique properties that are particularly suitable for mapping SNNs. Secondly, the Force Directed (FD) algorithm is developed to optimize the initial placement. The FD algorithm formulates the connections of clusters as tension forces, thus converts the local optimization of placement as a force analysis problem. The proposed approach is evaluated with the scale of 4 billion neurons, which is more than 200 times larger than previous research. The results show that our approach achieves state-of-the-art performance, significantly exceeding existing approaches.

References

[1]

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, and Michael Isard. 2016. Tensorflow: A system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). 265–283.

Digital Library

[2]

Adarsha Balaji, Prathyusha Adiraju, Hirak J Kashyap, Anup Das, Jeffrey L Krichmar, Nikil D Dutt, and Francky Catthoor. 2020. PyCARL: A PyNN interface for hardware-software co-simulation of spiking neural network. arXiv preprint arXiv:2003.09696.

[3]

Adarsha Balaji and Anup Das. 2019. A framework for the analysis of throughput-constraints of SNNs on neuromorphic hardware. In 2019 IEEE Computer Society Annual Symposium on VLSI (ISVLSI). 193–196.

[4]

Adarsha Balaji, Anup Das, Yuefeng Wu, Khanh Huynh, Francesco G. Dell’Anna, Giacomo Indiveri, Jeffrey L. Krichmar, Nikil D. Dutt, Siebren Schaafsma, and Francky Catthoor. 2020. Mapping Spiking Neural Networks to Neuromorphic Hardware. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 28, 1 (2020), 76–86. issn:1063-8210 https://doi.org/10.1109/tvlsi.2019.2951493

[5]

Ben Varkey Benjamin, Peiran Gao, Emmett McQuinn, Swadesh Choudhary, Anand R Chandrasekaran, Jean-Marie Bussat, Rodrigo Alvarez-Icaza, John V Arthur, Paul A Merolla, and Kwabena Boahen. 2014. Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations. Proc. IEEE, 102, 5 (2014), 699–716.

[6]

Anup Das, Yuefeng Wu, Khanh Huynh, Francesco Dell’Anna, Francky Catthoor, and Siebren Schaafsma. 2018. Mapping of local and global synapses on spiking neuromorphic hardware. In 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE). 1217–1222.

[7]

Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, and Shweta Jain. 2018. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38, 1 (2018), 82–99.

[8]

Michael V DeBole, Brian Taba, Arnon Amir, Filipp Akopyan, Alexander Andreopoulos, William P Risk, Jeff Kusnitz, Carlos Ortega Otero, Tapan K Nayak, and Rathinakumar Appuswamy. 2019. TrueNorth: Accelerating from zero to 64 million neurons in 10 years. Computer, 52, 5 (2019), 20–29.

[9]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. 248–255.

[10]

Li Deng. 2012. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29, 6 (2012), 141–142.

[11]

Shuiguang Deng, Pan Lv, Ouwen Jin, Schahram Dustdar, Ying Li, De Ma, Zhaohui Wu, and Gang Pan. 2022. Darwin-S: A Reference Software Architecture for Brain-Inspired Computers. Computer, 55, 5 (2022), 51–63. https://doi.org/10.1109/MC.2022.3144397

Digital Library

[12]

Steve B Furber, Francesco Galluppi, Steve Temple, and Luis A Plana. 2014. The spinnaker project. Proc. IEEE, 102, 5 (2014), 652–665.

[13]

Francesco Galluppi, Sergio Davies, Alexander Rast, Thomas Sharp, Luis A Plana, and Steve Furber. 2012. A hierachical configuration system for a massively parallel neural hardware platform. In Proceedings of the 9th conference on Computing Frontiers. 183–192.

Digital Library

[14]

Michael R Gary and David S Johnson. 1979. Computers and Intractability: A Guide to the Theory of NP-completeness.

[15]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.

[16]

David Hilbert. 1935. Dritter Band: Analysis · Grundlagen der Mathematik · Physik Verschiedenes, Nebst Einer Lebensgeschichte. 1–2. https://doi.org/10.1007/978-3-662-38452-7_1

[17]

Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

[18]

Yangfan Hu, Huajin Tang, and Gang Pan. 2021. Spiking Deep Residual Networks. IEEE Transactions on Neural Networks and Learning Systems, 1–6. https://doi.org/10.1109/TNNLS.2021.3119238

[19]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25 (2012).

[20]

Yann LeCun. 2015. LeNet-5, convolutional neural networks. URL: http://yann. lecun. com/exdb/lenet, 20, 5 (2015), 14.

[21]

Chenchen Liu, Bonan Yan, Chaofei Yang, Linghao Song, Zheng Li, Beiye Liu, Yiran Chen, Hai Li, Qing Wu, and Hao Jiang. 2015. A spiking neuromorphic design with resistive crossbar. In 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC). 1–6.

Digital Library

[22]

Qianhui Liu, Gang Pan, Haibo Ruan, Dong Xing, Qi Xu, and Huajin Tang. 2020. Unsupervised AER Object Recognition Based on Multiscale Spatio-Temporal Features and Spiking Neurons. IEEE Transactions on Neural Networks and Learning Systems, 31, 12 (2020), 5300–5311. https://doi.org/10.1109/TNNLS.2020.2966058

[23]

Xiaoxiao Liu, Wei Wen, Xuehai Qian, Hai Li, and Yiran Chen. 2018. Neu-NoC: A high-efficient interconnection network for accelerated neuromorphic systems. In 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC). 141–146.

Digital Library

[24]

De Ma, Juncheng Shen, Zonghua Gu, Ming Zhang, Xiaolei Zhu, Xiaoqiang Xu, Qi Xu, Yangjing Shen, and Gang Pan. 2017. Darwin: A neuromorphic hardware co-processor based on spiking neural networks. Journal of Systems Architecture, 77 (2017), 43–51.

[25]

Wolfgang Maass. 1997. Networks of spiking neurons: The third generation of neural network models. Neural Networks, 10, 9 (1997), 1659–1671. issn:0893-6080 https://doi.org/10.1016/s0893-6080(97)00011-7

[26]

Christian Mayr, Sebastian Hoeppner, and Steve Furber. 2019. Spinnaker 2: A 10 million core processor system for brain simulation and machine learning. arXiv preprint arXiv:1911.02385.

[27]

Carver Mead. 1990. Neuromorphic electronic systems. Proc. IEEE, 78, 10 (1990), 1629–1636.

[28]

Saber Moradi, Ning Qiao, Fabio Stefanini, and Giacomo Indiveri. 2017. A Scalable Multicore Architecture With Heterogeneous Memory Structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs). IEEE Transactions on Biomedical Circuits and Systems, 12, 1 (2017), 11, 106–122. issn:1932-4545 https://doi.org/10.1109/TBCAS.2017.2759700

[29]

Yibiao Rong, Xia Zhang, and Jianyu Lin. 2021. Modified Hilbert Curve for Rectangles and Cuboids and Its Application in Entropy Coding for Image and Video Compression. Entropy, 23, 7 (2021), 836. https://doi.org/10.3390/e23070836

[30]

Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. 2019. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575, 7784 (2019), 607–617. issn:0028-0836 https://doi.org/10.1038/s41586-019-1677-2

[31]

Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. 2017. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in neuroscience, 11 (2017), 682.

[32]

Pradip Kumar Sahu and Santanu Chattopadhyay. 2013. A survey on application mapping strategies for Network-on-Chip design. Journal of Systems Architecture, 59, 1 (2013), 60–76. issn:1383-7621 https://doi.org/10.1016/j.sysarc.2012.10.004

Digital Library

[33]

Jun Sawada, Filipp Akopyan, Andrew S Cassidy, Brian Taba, Michael V Debole, Pallab Datta, Rodrigo Alvarez-Icaza, Arnon Amir, John V Arthur, and Alexander Andreopoulos. 2016. Truenorth ecosystem for brain-inspired computing: scalable systems, software, and applications. In SC’16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 130–141.

[34]

Johannes Schemmel. 2021. The BrainScaleS accelerated analogue neuromorphic architecture. Brain-inspired Computing, 14.

[35]

Luping Shi, Jing Pei, Ning Deng, Dong Wang, Lei Deng, Yu Wang, Youhui Zhang, Feng Chen, Mingguo Zhao, and Sen Song. 2015. Development of a neuromorphic computing system. In 2015 IEEE international electron devices meeting (IEDM). 4–3.

[36]

Shihao Song, Harry Chong, Adarsha Balaji, Anup Das, James Shackleford, and Nagarajan Kandasamy. 2022. DFSynthesizer: Dataflow-based synthesis of spiking neural networks to neuromorphic hardware. ACM Transactions on Embedded Computing Systems (TECS), 21, 3 (2022), 1–35.

Digital Library

[37]

Shihao Song, M Lakshmi Varshika, Anup Das, and Nagarajan Kandasamy. 2021. A design flow for mapping spiking neural networks to many-core neuromorphic hardware. In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD). 1–9.

Digital Library

[38]

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. 2015. Rethinking the Inception Architecture for Computer Vision. CoRR, abs/1512.00567 (2015), arxiv:1512.00567. arxiv:1512.00567

[39]

Giulio Tononi, Olaf Sporns, and Gerald M Edelman. 1994. A measure for brain complexity: relating functional segregation and integration in the nervous system. Proceedings of the National Academy of Sciences, 91, 11 (1994), 5033–5037.

Cited By

Mustafazade IKandasamy NDas A(2024)Clustering and Allocation of Spiking Neural Networks on Crossbar-Based Neuromorphic ArchitectureProceedings of the 21st ACM International Conference on Computing Frontiers10.1145/3649153.3649199(164-171)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3649153.3649199
Xiao CHe XYang ZXiao XWang YGong RTie JWang LXu W(2024)Hierarchical Mapping of Large-Scale Spiking Convolutional Neural Networks Onto Resource-Constrained Neuromorphic ProcessorIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2023.334407043:5(1442-1455)Online publication date: May-2024
https://doi.org/10.1109/TCAD.2023.3344070
Titirsha THossain Shuvo MAkter SIslam S(2024)Hill Climbing for Efficient Spiking Neural Network Acceleration on Neuromorphic Chips2024 IEEE 67th International Midwest Symposium on Circuits and Systems (MWSCAS)10.1109/MWSCAS60917.2024.10658659(932-936)Online publication date: 11-Aug-2024
https://doi.org/10.1109/MWSCAS60917.2024.10658659
Show More Cited By

Index Terms

Mapping Very Large Scale Spiking Neuron Network to Neuromorphic Hardware

Recommendations

Run-time Mapping of Spiking Neural Networks to Neuromorphic Hardware
Abstract
Neuromorphic architectures implement biological neurons and synapses to execute machine learning algorithms with spiking neurons and bio-inspired learning algorithms. These architectures are energy efficient and therefore, suitable for cognitive ...
Effects of Noise on Leaky Integrate-and-Fire Neuron Models for Neuromorphic Computing Applications
Computational Science and Its Applications – ICCSA 2022
Abstract
Artificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), ...
DFSynthesizer: Dataflow-based Synthesis of Spiking Neural Networks to Neuromorphic Hardware
Spiking Neural Networks (SNNs) are an emerging computation model that uses event-driven activation and bio-inspired learning algorithms. SNN-based machine learning programs are typically executed on tile-based neuromorphic hardware platforms, where each ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ASPLOS 2023: Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3

March 2023

820 pages

ISBN:9781450399180

DOI:10.1145/3582016

General Chair:
Tor M. Aamodt
University of British Columbia, Canada
,
Program Chairs:
Natalie Enright Jerger
University of Toronto, Canada
,
Michael Swift
University of Wisconsin-Madison, USA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 March 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key Research and Development Program of China

Conference

ASPLOS '23

Sponsor:

ASPLOS '23: 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3

March 25 - 29, 2023

BC, Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
901
Total Downloads

Downloads (Last 12 months)342
Downloads (Last 6 weeks)27

Reflects downloads up to 16 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mustafazade IKandasamy NDas A(2024)Clustering and Allocation of Spiking Neural Networks on Crossbar-Based Neuromorphic ArchitectureProceedings of the 21st ACM International Conference on Computing Frontiers10.1145/3649153.3649199(164-171)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3649153.3649199
Xiao CHe XYang ZXiao XWang YGong RTie JWang LXu W(2024)Hierarchical Mapping of Large-Scale Spiking Convolutional Neural Networks Onto Resource-Constrained Neuromorphic ProcessorIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2023.334407043:5(1442-1455)Online publication date: May-2024
https://doi.org/10.1109/TCAD.2023.3344070
Titirsha THossain Shuvo MAkter SIslam S(2024)Hill Climbing for Efficient Spiking Neural Network Acceleration on Neuromorphic Chips2024 IEEE 67th International Midwest Symposium on Circuits and Systems (MWSCAS)10.1109/MWSCAS60917.2024.10658659(932-936)Online publication date: 11-Aug-2024
https://doi.org/10.1109/MWSCAS60917.2024.10658659
Ma DJin XSun SLi YWu XHu YYang FTang HZhu XLin PPan G(2024)Darwin3: a large-scale neuromorphic chip with a novel ISA and on-chip learningNational Science Review10.1093/nsr/nwae10211:5Online publication date: 18-Mar-2024
https://doi.org/10.1093/nsr/nwae102
Huang LLv PDu XJin ODeng S(2024)A Hierarchical Neural Task Scheduling Algorithm in the Operating System of Neuromorphic ComputersKnowledge Science, Engineering and Management10.1007/978-981-97-5501-1_11(135-150)Online publication date: 27-Jul-2024
https://doi.org/10.1007/978-981-97-5501-1_11
Xie LXue JWu LChen FTian QZhou YYing RLiu P(2023)SpikeNC: An Accurate and Scalable Simulator for Spiking Neural Network on Multi-Core Neuromorphic Hardware2023 IEEE 30th International Conference on High Performance Computing, Data, and Analytics (HiPC)10.1109/HiPC58850.2023.00052(357-366)Online publication date: 18-Dec-2023
https://doi.org/10.1109/HiPC58850.2023.00052
Dang KDoan NNguyen NAbdallah A(2023)HeterGenMap: An Evolutionary Mapping Framework for Heterogeneous NoC-Based Neuromorphic SystemsIEEE Access10.1109/ACCESS.2023.334516811(144095-144112)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3345168
Yerima WDang KAbdallah A(2023)R-MaS3N: Robust Mapping of Spiking Neural Networks to 3D-NoC-Based Neuromorphic Systems for Enhanced ReliabilityIEEE Access10.1109/ACCESS.2023.331103111(94664-94678)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3311031

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents