More Web Proxy on the site http://driver.im/

research-article

Fault tolerance in memristive crossbar-based neuromorphic computing systems

Authors:

Zhengfeng HuangAuthors Info & Claims

Volume 70, Issue C

Pages 70 - 79

https://doi.org/10.1016/j.vlsi.2019.09.008

Published: 01 January 2020 Publication History

Abstract

In recent years, neuromorphic computing systems (NCS) based on memristive crossbar have provided a promising solution to enable acceleration of neural networks. However, Stuck-at faults in the memristor devices significantly degrade the computing accuracy of NCS. In this paper, we propose an effective fault tolerant framework for memristive crossbar-based neuromorphic computing systems. First, a fault tolerance-aware hierarchical clustering method is proposed to partition weight connections of a sparse neural network into clusters. Then, for each cluster, memristive crossbar configuration is proposed to determine a suitable size of the crossbar with consideration of both hardware cost and successful mapping rate. Next, an integer linear programming formulation is developed to derive a connection-memristor mapping for fault tolerance. Finally, an efficient matching-based heuristic algorithm is further proposed to speed-up the fault-tolerant mapping process. Experimental results show that the proposed fault tolerant framework can improve the successful mapping rate and simultaneously reduce the hardware cost.

Highlights

•

Given a sparse neural network, a fault tolerance-aware hierarchical clustering method is proposed to partition weight connections into a set of clusters.

•

For connection matrix of each cluster, a non-linear programming is proposed to determine suitable size of the mapped memristive crossbar, considering both hardware cost and successful mapping rate.

•

Fault-tolerant mapping is formulated as an integer linear programming (ILP) to map connection matrix to memristive crossbar.

•

An efficient matching-based heuristic algorithm is further proposed to speed-up the ILP process.

•

A Monte Carlo simulation is exploited to evaluate the performance of the fault tolerant synapse mapping framework on different benchmarks.

References

[1]

Y. Chen, H.H. Li, C. Wu, C. Song, S. Li, C. Min, H.-P. Cheng, W. Wen, X. Liu, Neuromorphic computing's yesterday, today, and tomorrowan evolutional view, Integr. VLSI J. 61 (2018) 49–61.

[2]

W. Wen, C.-R. Wu, X. Hu, B. Liu, T.-Y. Ho, X. Li, Y. Chen, An EDA framework for large scale hybrid neuromorphic computing systems, in: ACM/IEEE Design Automation Conference (DAC), 2015, pp. 1–6.

[3]

Q. Xu, S. Chen, B. Yu, F. Wu, Memristive crossbar mapping for neuromorphic computing systems on 3D IC, in: ACM Great Lakes Symposium on VLSI (GLSVLSI), 2018, pp. 451–454.

[4]

S. Acciarito, A. Cristini, G. Susi, et al., Hardware design of lif with latency neuron model with memristive stdp synapses, Integr. VLSI J. 59 (2017) 81–89.

[5]

Z. Li, C. Liu, Y. Wang, B. Yan, C. Yang, J. Yang, H. Li, An overview on memristor crossabr based neuromorphic circuit and architecture, in: IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SOC), 2015, pp. 52–56.

[6]

C.-R. Wu, W. Wen, T.-Y. Ho, Y. Chen, Thermal optimization for memristor-based hybrid neuromorphic computing systems, in: IEEE/ACM Asia and South Pacific Design Automation Conference (ASPDAC), 2016, pp. 274–279.

[7]

J. Cui, Q. Qiu, Towards memristor based accelerator for sparse matrix vector multiplication, in: IEEE International Symposium on Circuits and Systems (ISCAS), 2016, pp. 121–124.

[8]

W. Huangfu, L. Xia, M. Cheng, X. Yin, T. Tang, B. Li, K. Chakrabarty, Y. Xie, Y. Wang, H. Yang, Computation-oriented fault-tolerance schemes for RRAM computing systems, in: IEEE/ACM Asia and South Pacific Design Automation Conference (ASPDAC), 2017, pp. 794–799.

[9]

L. Xia, M. Liu, X. Ning, K. Chakrabarty, Y. Wang, Fault-tolerant training with on-line fault detection for rram-based neural computing systems, ACM/IEEE Design Automation Conference (DAC), vol. 33, 2017.

[10]

B. Yuan, B. Li, T. Weise, X. Yao, A new memetic algorithm with fitness approximation for the defect-tolerant logic mapping in crossbar-based nanoarchitectures, IEEE Trans. Evol. Comput. 18 (2014) 846–859.

[11]

O. Tunali, M. Altun, Logic synthesis and defect tolerance for memristive crossbar arrays, in: IEEE/ACM Proceedings Design, Automation and Test in Eurpoe (DATE), 2018, pp. 425–430.

[12]

Y. Su, W. Rao, An integrated framework toward defect-tolerant logic implementation onto nanocrossbars, IEEE Trans. Comput. Aided Des. Integr Circuits Syst. 33 (2014) 64–75.

[13]

O. Tunali, M. Altun, A fast logic mapping algorithm for multiple-type-defect tolerance in reconfigurable nano-crossbar arrays, IEEE Trans. Emerg. Top. Comput. (2017),.

[14]

L. Chen, J. Li, Y. Chen, Q. Deng, J. Shen, X. Liang, L. Jiang, Accelerator-friendly neural-network training: learning variations and defects in RRAM crossbar, in: IEEE/ACM Proceedings Design, Automation and Test in Eurpoe (DATE), 2017, pp. 19–24.

[15]

C. Liu, M. Hu, J.P. Strachan, H. Li, Rescuing memristor-based neuromorphic design with high defects, in: ACM/IEEE Design Automation Conference (DAC), 2017, pp. 1–6.

[16]

S. Kim, P. Howe, T. Moreau, A. Alaghi, L. Ceze, V.S. Sathe, Energy-efficient neural network acceleration in the presence of bit-level memory errors, IEEE Trans. Circuits Syst. I (2018) 1–14.

[17]

L. Xia, W. Huangfu, T. Tang, X. Yin, K. Chakrabarty, Y. Xie, Y. Wang, H. Yang, Stuck-at fault tolerance in RRAM computing systems, IEEE Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS) 8 (2018) 102–115.

[18]

Y. Wang, W. Wen, L. Song, H.H. Li, Classification accuracy improvement for neuromorphic computing systems with one-level precision synapses, in: IEEE/ACM Asia and South Pacific Design Automation Conference (ASPDAC), 2017, pp. 776–781.

[19]

M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, Y. Bengio, Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to 1 or -1, 2016, arXiv preprint arXiv:1602.02830.

[20]

O. Tunali, M.C. Morgul, M. Altun, Defect-tolerant logic synthesis for memristor crossbars with performance evaluation, IEEE Micro 38 (2018) 22–31.

[21]

C. Li, D. Belkin, Y. Li, P. Yan, M. Hu, N. Ge, H. Jiang, E. Montgomery, P. Lin, Z. Wang, et al., Efficient and self-adaptive in-situ learning in multilayer memristor neural networks, Nat. Commun. 9 (2018) 2385.

[22]

C.-Y. Chen, H.-C. Shih, C.-W. Wu, C.-H. Lin, P.-F. Chiu, S.-S. Sheu, F.T. Chen, RRAM defect modeling and failure analysis based on march test and a novel squeeze-search scheme, IEEE Trans. Comput. (2014) 1.

[23]

B. Li, P. Gu, Y. Shan, Y. Wang, Y. Chen, H. Yang, RRAM-based analog approximate computing, IEEE Trans. Comput. Aided Des. Integr Circuits Syst. 34 (2015) 1905–1917.

[24]

S. Kannan, J. Rajendran, R. Karri, O. Sinanoglu, Sneak-path testing of crossbar-based nonvolatile random access memories, IEEE Trans. Nanotechnol. 12 (2013) 413–426.

[25]

S. Chen, T. Yoshimura, Fixed-outline floorplanning: enumerating block positions and a new objective function for calculating area costs, IEEE Trans. Comput. Aided Des. Integr Circuits Syst. 27 (2008) 858–871.

[26]

R.C. Dubes, A.K. Jain, Algorithms for Clustering Data, Prentice Hall Englewood Cliffs, New Jersey, 1988.

[27]

S. Salvador, P. Chan, Determining the number of clusters/segments in hierarchical clustering/segmentation algorithms, IEEE International Conference on Tools with Artificial Intelligence, 2004, pp. 576–584.

[28]

O. Tunali, M. Altun, Permanent and transient fault tolerance for reconfigurable nano-crossbar arrays, IEEE Trans. Comput. Aided Des. Integr Circuits Syst. 36 (2017) 747–760.

[29]

O. Tunali, M. Altun, A survey of fault-tolerance algorithms for reconfigurable nano-crossbar arrays, ACM Comput. Surv. 50 (2018) 79.

[30]

M.R. Garey, D.S. Johnson, Computers and Intractability: A Guide to NP-Completeness, WH Freeman and Company, San Francisco, 1979.

[31]

J. Munkres, Algorithms for the assignment and transportation problems, J. Soc. Ind. Appl. Math. 5 (1957) 32–38.

[32]

Gurobi Optimization Inc. (2014): Gurobi optimizer reference manual. http://www.gurobi.com.

[33]

B. Liu, Y. Chen, B. Wysocki, T. Huang, The circuit realization of a neuromorphic computing system with memristor-based synapse design, in: Springer International Conference on Neural Information Processing (ICONIP), 2012, pp. 357–365.

[34]

A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, in: Conference on Neural Information Processing Systems (NIPS), 2012, pp. 1097–1105.

Cited By

Ma YZheng LZhou P(2023)A Mapping Method Tolerating SAF and Variation for Memristor Crossbar Array Based Neural Network Inference on Edge DevicesACM Journal on Emerging Technologies in Computing Systems10.1145/358551819:2(1-21)Online publication date: 25-Feb-2023
https://dl.acm.org/doi/10.1145/3585518
Huang CXu NZeng JWang WHu YFang LMa DChen Y(2022)Rescuing ReRAM-based Neural Computing Systems from Device VariationACM Transactions on Design Automation of Electronic Systems10.1145/353370628:1(1-17)Online publication date: 10-Dec-2022
https://dl.acm.org/doi/10.1145/3533706
Ansari ASharma VMishra R(2022)A 3-disjoint path design of non-blocking shuffle exchange network by extra port alignmentThe Journal of Supercomputing10.1007/s11227-022-04450-278:12(14381-14401)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1007/s11227-022-04450-2
Show More Cited By

Index Terms

Fault tolerance in memristive crossbar-based neuromorphic computing systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Fault Tolerance in Multiprocessor Systems Without Dedicated Redundancy

An algorithm called RAFT (recursive algorithm for fault tolerance) for achieving fault tolerance in multiprocessor systems is described. Through the use of a combination of dynamic space- and time- redundancy techniques, RAFT achieves fault tolerance in ...
Memristive Crossbar Mapping for Neuromorphic Computing Systems on 3D IC

In recent years, neuromorphic computing systems based on memristive crossbar have provided a promising solution to enable acceleration of neural networks. However, most of the neural networks used in realistic applications are often sparse. If such ...
Fault Tolerance of Multiple Logic Faults in SRAM-Based FPGA Systems
DSD '11: Proceedings of the 2011 14th Euromicro Conference on Digital System Design

The very hight levels of integration and submicron device sizes used in current and emerging VLSI technologies for SRAM-based FPGAs lead to higher occurrences of defects and operational faults. Thus, there is a critical need for fault tolerance and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Integration, the VLSI Journal

Integration, the VLSI Journal Volume 70, Issue C

Jan 2020

160 pages

ISSN:0167-9260

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 January 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ma YZheng LZhou P(2023)A Mapping Method Tolerating SAF and Variation for Memristor Crossbar Array Based Neural Network Inference on Edge DevicesACM Journal on Emerging Technologies in Computing Systems10.1145/358551819:2(1-21)Online publication date: 25-Feb-2023
https://dl.acm.org/doi/10.1145/3585518
Huang CXu NZeng JWang WHu YFang LMa DChen Y(2022)Rescuing ReRAM-based Neural Computing Systems from Device VariationACM Transactions on Design Automation of Electronic Systems10.1145/353370628:1(1-17)Online publication date: 10-Dec-2022
https://dl.acm.org/doi/10.1145/3533706
Ansari ASharma VMishra R(2022)A 3-disjoint path design of non-blocking shuffle exchange network by extra port alignmentThe Journal of Supercomputing10.1007/s11227-022-04450-278:12(14381-14401)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1007/s11227-022-04450-2
Yadav DThangkhiew PDatta KChakraborty SDrechsler RSengupta I(2022)FAMCroNA: Fault Analysis in Memristive Crossbars for Neuromorphic ApplicationsJournal of Electronic Testing: Theory and Applications10.1007/s10836-022-06001-238:2(145-163)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s10836-022-06001-2
Ma YZheng LZhou PChen YZhirnov VSasan ASavidis I(2021)Tolerating Stuck-at Fault and Variation in Resistive Edge Inference Engine via Weight MappingProceedings of the 2021 Great Lakes Symposium on VLSI10.1145/3453688.3461487(313-318)Online publication date: 22-Jun-2021
https://dl.acm.org/doi/10.1145/3453688.3461487

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents