More Web Proxy on the site http://driver.im/

research-article

Logic Synthesis of Binarized Neural Networks for Efficient Circuit Implementation

Authors:

Jie-Hong R. JiangAuthors Info & Claims

2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD)

Pages 1 - 7

https://doi.org/10.1145/3240765.3240822

Published: 05 November 2018 Publication History

Abstract

Neural networks (NNs) are key to deep learning systems. Their efficient hardware implementation is crucial to applications at the edge. Binarized NNs (BNNs), where the weights and output of a neuron are of binary values {–1, +1} (or encoded in {0, 1}), have been proposed recently. As no multiplier is required, they are particularly attractive and suitable for hardware realization. Most prior NN synthesis methods target on hardware architectures with neural processing elements (NPEs), where the weights of a neuron are loaded and the output of the neuron is computed. The load-and-compute method, though area efficient, requires expensive memory access, which deteriorates energy and performance efficiency. In this work we aim at synthesizing BNN dense layers into dedicated logic circuits. We formulate the corresponding matrix covering problem and propose a scalable algorithm to reduce the area and routing cost of BNNs. Experimental results justify the effectiveness of the method in terms of area and net savings on FPGA implementation. Our method provides an alternative implementation of BNNs, and can be applied in combination with NPE-based implementation for area, speed, and power tradeoffs.

References

[1]

H. Alemdar, V. Leroy, A. Prost-Boucle, and F. Pétrot Ternary Neural Networks for Resource-Efficient AI Applications. arXiv e-print, arXiv:, 2016.

[2]

M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y. Bengio. Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or −1. arXiv e-print, arXiv:, 2016.

[3]

N. Fraser, Y. Umuroglu, G. Gambardella, M. Blott, P. Leong, M. Jahre, and K. Vissers. Scaling Binarized Neural Networks on Reconfigurable Logic. arXiv e-print, arXiv:, 2017.

[4]

I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning, MIT Press, 2016.

Digital Library

[5]

S. Han, H. Mao, and W. Dally Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. In Proc. International Conference on Learning Representations (ICLR), 2016.

[6]

P. Hart, N. Nilsson, and B. Raphael. A Formal Basis for the Heuristic Determination of Minimum Cost Paths. IEEE Transactions on Systems Science and Cybernetics, 4 (2): 100–107, 1968.

[7]

K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, 2015.

[8]

M. Horowitz. Computing's energy problem (and what we can do about it). In Proc. International Solid-State Circuits Conference (ISSCC), pp. 10–14, 2014.

[9]

A. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv e-print, arXiv:, 2017.

[10]

A. Krizhevsky. Learning Multiple Layers of Features from Tiny Images. MS thesis, University of Toronto, https://www.cs.toronto.edu/~kriz/cifar.html, 2009.

[11]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86 (11): 2278–2324, 1998.

[12]

Y. LeCun, C. Cortes, and C. Burges. THE MNIST DATABASE of handwritten digits. http://yann.lecun.com/exdb/mnist/, 1998.

[13]

M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. arXiv e-print, arXiv:, 2016.

[14]

R. Rudell Logic Synthesis for VLSI Design. PhD dissertation, University of California, Berkeley, 1989.

Digital Library

[15]

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, and S. Reed. Going Deeper with Convolutions. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9, 2015.

[16]

Y. Umuroglu, N. Fraser, G. Gambardella, M. Blott, P. Leong, M. Jahre, and K. Vissers FINN: A Framework for Fast, Scalable Binarized Neural Network Inference. In Proc. International Symposium on Field-Programmable Gate Arrays (FPGA), pp. 65–74, 2017.

[17]

H. Yonekawa and H. Nakahara On-Chip Memory Based Binarized Convolutional Deep Neural Network Applying Batch Normalization Free Technique on an FPGA. In Proc. International Parallel and Distributed Processing Symposium Workshops (IPDPS), pp. 98–105, 2017.

[18]

R. Zhao, W. Song, W. Zhang, T. Xing, J.-H. Lin, M. Srivastava, R. Gupta, and Z. Zhang Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs. In Proc. International Symposium on Field-Programmable Gate Arrays (FPGA), pp. 15–24, 2017.

Cited By

Smets LVan Leekwijck WTsang ILatré S(2024)An encoding framework for binarized images using hyperdimensional computingFrontiers in Big Data10.3389/fdata.2024.13715187Online publication date: 14-Jun-2024
https://doi.org/10.3389/fdata.2024.1371518
Huang YJiang JMishchenko A(2023)Quantized Neural Network Synthesis for Direct Logic Circuit ImplementationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.318354742:2(473-482)Online publication date: Feb-2023
https://doi.org/10.1109/TCAD.2022.3183547
Chi CJiang J(2022)Logic Synthesis of Binarized Neural Networks for Efficient Circuit ImplementationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2021.307860641:4(993-1005)Online publication date: Apr-2022
https://doi.org/10.1109/TCAD.2021.3078606
Show More Cited By

Index Terms

Logic Synthesis of Binarized Neural Networks for Efficient Circuit Implementation
1. Hardware
  1. Integrated circuits

Index terms have been assigned to the content through auto-classification.

Recommendations

Binarized neural networks
NIPS'16: Proceedings of the 30th International Conference on Neural Information Processing Systems

We introduce a method to train Binarized Neural Networks (BNNs) - neural networks with binary weights and activations at run-time. At train-time the binary weights and activations are used for computing the parameter gradients. During the forward pass, ...
Multi-level neural networks, their applications and VLSI circuit implementation
Scaling Binarized Neural Networks on Reconfigurable Logic
PARMA-DITAM '17: Proceedings of the 8th Workshop and 6th Workshop on Parallel Programming and Run-Time Management Techniques for Many-core Architectures and Design Tools and Architectures for Multicore Embedded Computing Platforms

Binarized neural networks (BNNs) are gaining interest in the deep learning community due to their significantly lower computational and memory cost. They are particularly well suited to reconfigurable logic devices, which contain an abundance of fine-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD)

Nov 2018

939 pages

Copyright © 2018.

Publisher

IEEE Press

Publication History

Published: 05 November 2018

Permissions

Request permissions for this article.

Request Permissions

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
356
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Smets LVan Leekwijck WTsang ILatré S(2024)An encoding framework for binarized images using hyperdimensional computingFrontiers in Big Data10.3389/fdata.2024.13715187Online publication date: 14-Jun-2024
https://doi.org/10.3389/fdata.2024.1371518
Huang YJiang JMishchenko A(2023)Quantized Neural Network Synthesis for Direct Logic Circuit ImplementationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.318354742:2(473-482)Online publication date: Feb-2023
https://doi.org/10.1109/TCAD.2022.3183547
Chi CJiang J(2022)Logic Synthesis of Binarized Neural Networks for Efficient Circuit ImplementationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2021.307860641:4(993-1005)Online publication date: Apr-2022
https://doi.org/10.1109/TCAD.2021.3078606
Chen TKemp TKim Y(2022)SYNTHNET: A High-throughput yet Energy-efficient Combinational Logic Neural Network2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC)10.1109/ASP-DAC52403.2022.9712554(232-237)Online publication date: 17-Jan-2022
https://doi.org/10.1109/ASP-DAC52403.2022.9712554
Santikellur PChakraborty RSantikellur PChakraborty R(2022)Combinational Logic-Based Implementation of PUFDeep Learning for Computational Problems in Hardware Security10.1007/978-981-19-4017-0_6(71-82)Online publication date: 16-Sep-2022
https://doi.org/10.1007/978-981-19-4017-0_6
Vreca JIvanov IPapa GBiasizzo A(2021)Detecting Network Intrusion Using Binarized Neural Networks2021 IEEE 7th World Forum on Internet of Things (WF-IoT)10.1109/WF-IoT51360.2021.9595961(622-627)Online publication date: 14-Jun-2021
https://doi.org/10.1109/WF-IoT51360.2021.9595961
Lebedev MBelecky P(2021)A Survey of Open-source Tools for FPGA-based Inference of Artificial Neural Networks2021 Ivannikov Memorial Workshop (IVMEM)10.1109/IVMEM53963.2021.00015(50-56)Online publication date: Sep-2021
https://doi.org/10.1109/IVMEM53963.2021.00015
Liu QGao JLai J(2021)TCP-Net: Minimizing Operation Counts of Binarized Neural Network Inference2021 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS51556.2021.9401549(1-5)Online publication date: May-2021
https://doi.org/10.1109/ISCAS51556.2021.9401549
Zeng WDavoodi ATopaloglu R(2021)Sampling-Based Approximate Logic Synthesis: An Explainable Machine Learning Approach2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD)10.1109/ICCAD51958.2021.9643484(1-9)Online publication date: 1-Nov-2021
https://doi.org/10.1109/ICCAD51958.2021.9643484
Murovic TTrost A(2021)Massively parallel binary neural network inference for detecting ships in FPGA systems on the edge2021 24th Euromicro Conference on Digital System Design (DSD)10.1109/DSD53832.2021.00023(90-96)Online publication date: Sep-2021
https://doi.org/10.1109/DSD53832.2021.00023
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents