More Web Proxy on the site http://driver.im/

research-article

An Efficient Reinforcement Learning Based Framework for Exploring Logic Synthesis

Authors:

Lingli WangAuthors Info & Claims

ACM Transactions on Design Automation of Electronic Systems, Volume 29, Issue 2

Article No.: 25, Pages 1 - 33

https://doi.org/10.1145/3632174

Published: 15 January 2024 Publication History

Abstract

Logic synthesis is a crucial step in electronic design automation tools. The rapid developments of reinforcement learning (RL) have enabled the automated exploration of logic synthesis. Existing RL based methods may lead to data inefficiency, and the exploration approaches for FPGA and ASIC technology mapping in recent works lack the flexibility of the learning process. This work proposes ESE, a reinforcement learning based framework to efficiently learn the logic synthesis process. The framework supports the modeling of logic optimization and technology mapping for FPGA and ASIC. The optimization for the execution time of the synthesis script is also considered. For the modeling of FPGA mapping, the logic optimization and technology mapping are combined to be learned in a flexible way. For the modeling of ASIC mapping, the standard cell based optimization and LUT optimization operations are incorporated into the ASIC synthesis flow. To improve the utilization of samples, the Proximal Policy Optimization model is adopted. Furthermore, the framework is enhanced by supporting MIG based synthesis exploration. Experiments show that for FPGA technology mapping on the VTR benchmark, the average LUT-Level-Product and script runtime are improved by more than 18.3% and 12.4% respectively than previous works. For ASIC mapping on the EPFL benchmark, the average Area-Delay-Product is improved by 14.5%.

References

[1]

David Abrahams and Ralf W. Grosse-Kunstleve. 2003. Building hybrid systems with Boost. Python. C/C++ Users Journal 21, LBNL-53142 (2003), 1-13.

[2]

Luca Amarú, Pierre-Emmanuel Gaillardon, and Giovanni De Micheli. 2014. Majority-inverter graph: A novel data-structure and algorithms for efficient logic optimization. In Proceedings of the 2014 51st ACM/EDAC/IEEE Design Automation Conference. IEEE, 1–6.

Digital Library

[3]

Luca Amarú, Pierre-Emmanuel Gaillardon, and Giovanni De Micheli. 2015. The EPFL combinational benchmark suite. In Proceedings of the 24th International Workshop on Logic and Synthesis (IWLS’15).

[4]

Luca Amarú, Vinicius Possani, Eleonora Testa, Felipe Marranghello, Christopher Casares, Jiong Luo, Patrick Vuillod, Alan Mishchenko, and Giovanni De Micheli. 2021. LUT-based optimization for ASIC design flow. In Proceedings of the 2021 58th ACM/IEEE Design Automation Conference. IEEE, 871–876.

Digital Library

[5]

Max Austin, Scott Temple, Walter Lau Neto, Luca Amarù, Xifan Tang, and Pierre-Emmanuel Gaillardon. 2020. A scalable mixed synthesis framework for heterogeneous networks. In Proceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition. IEEE, 670–673.

[6]

Franc Brglez, David Bryan, and Krzysztof Kozminski. 1989. Combinational profiles of sequential benchmark circuits. In Proceedings of the IEEE International Symposium on Circuits and Systems. IEEE, 1929–1934.

[7]

Hongzheng Chen and Minghua Shen. 2019. A deep-reinforcement-learning-based scheduler for high-level synthesis. In Proceedings of the ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. ACM, 117–117.

Digital Library

[8]

Ruoyu Cheng and Junchi Yan. 2021. On joint learning for solving placement and routing in chip design. In Proceedings of the Advances in Neural Information Processing Systems.

[9]

Brendon Chetwynd, Kevin Bush, and Kyle Ingols. [n. d.]. Common Evaluation Platform. Retrieved from https://github.com/mit-ll/CEP.git. Accessed 2022.

[10]

Animesh Basak Chowdhury, Benjamin Tan, Ryan Carey, Tushit Jain, Ramesh Karri, and Siddharth Garg. 2022. Bulls-Eye: Active few-shot learning guided logic synthesis. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 42, 8 (2022), 2580-2590.

[11]

Zhufei Chu. [n. d.]. Advanced Logic Synthesis and Optimization Tool (ALSO). Retrieved from https://gitee.com/zfchu/also. Accessed 2021.

[12]

Zhufei Chu, Mathias Soeken, Yinshui Xia, Lunyao Wang, and Giovanni De Micheli. 2019. Structural rewriting in XOR-majority graphs. In Proceedings of the 24th Asia and South Pacific Design Automation Conference. ACM, 663–668.

Digital Library

[13]

Lawrence T. Clark, Vinay Vashishtha, Lucian Shifren, Aditya Gujja, Saurabh Sinha, Brian Cline, Chandarasekaran Ramamurthy, and Greg Yeric. 2016. ASAP7: A 7-nm finFET predictive process design kit. Microelectronics Journal 53 (2016), 105–115.

Digital Library

[14]

Winston Haaswijk, Edo Collins, Benoit Seguin, Mathias Soeken, Frédéric Kaplan, Sabine Süsstrunk, and Giovanni De Micheli. 2018. Deep learning for logic optimization algorithms. In Proceedings of the IEEE International Symposium on Circuits and Systems. 1–4.

[15]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.

Digital Library

[16]

Abdelrahman Hosny, Soheil Hashemi, Mohamed Shalan, and Sherief Reda. 2020. DRiLLS: Deep reinforcement learning for logic synthesis. In Proceedings of the IEEE 25th Asia and South Pacific Design Automation Conference. IEEE, 581–586.

Digital Library

[17]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR'15). 1-13.

[18]

Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations (ICLR'17), 1-14.

[19]

Kun Kong, Yun Shang, and Ruqian Lu. 2009. An optimized majority logic synthesis methodology for quantum-dot cellular automata. IEEE Transactions on Nanotechnology 9, 2 (2009), 170–183.

Digital Library

[20]

Hongzi Mao, Mohammad Alizadeh, Ishai Menache, and Srikanth Kandula. 2016. Resource management with deep reinforcement learning. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks. ACM, 50–56.

Digital Library

[21]

Giovanni De Micheli. 1994. Synthesis and Optimization of Digital Circuits. McGraw-Hill Higher Education.

Digital Library

[22]

Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Sungmin Bae, Azade Nazi, Jiwoo Pak, Andy Tong, Kavya Srinivasa, William Hang, Emre Tuncer, Anand Babu, Quoc V. Le, James Laudon, Richard Ho, Roger Carpenter, and Jeff Dean. 2020. Chip placement with deep reinforcement learning. arXiv:2004.10746. Retrieved from https://arxiv.org/abs/2004.10746

[23]

Alan Mishchenko and Robert Brayton. 2007. ABC: A System for Sequential Synthesis and Verification. Retrieved from http://www.eecs.berkeley.edu/alanmi/abc. Accessed 2021.

[24]

Alan Mishchenko, Satrajit Chatterjee, and Robert Brayton. 2006. DAG-aware AIG rewriting: A fresh look at combinational logic synthesis. In Proceedings of the 43rd ACM/IEEE Design Automation Conference. ACM, 532–535.

Digital Library

[25]

Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In Proceedings of the International Conference on Machine Learning. PMLR, 1928–1937.

Digital Library

[26]

Kevin E. Murray, Oleg Petelin, Sheng Zhong, Jai Min Wang, Mohamed ElDafrawy, Jean-Philippe Legault, Eugene Sha, Aaron G. Graham, Jean Wu, Matthew J. P. Walker, Hanqing Zeng, Panagiotis Patros, Jason Luu, Kenneth B. Kent, and Vaughn Betz. 2020. VTR 8: High performance CAD and customizable FPGA architecture modelling. ACM Transactions on Reconfigurable Technology and Systems 13, 2 (2020), 1-55.

Digital Library

[27]

Walter Lau Neto, Max Austin, Scott Temple, Luca Amaru, Xifan Tang, and Pierre-Emmanuel Gaillardon. 2019. LSOracle: A logic synthesis framework driven by artificial intelligence. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. IEEE, 1–6.

[28]

Walter Lau Neto, Yingjie Li, Pierre-Emmanuel Gaillardon, and Cunxi Yu. 2023. FlowTune: End-to-end automatic logic optimization exploration via domain-specific multiarmed bandit. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 42, 6 (2023), 1912–1925.

Digital Library

[29]

Art B. Owen. 2013. Monte Carlo Theory, Methods and Examples. https://artowen.su.domains/mc/

[30]

Peichen Pan, Arvind K. Karandikar, and C. L. Liu. 1998. Optimal clock period clustering for sequential circuits with retiming. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 17, 6 (1998), 489–498.

Digital Library

[31]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. In Proceedings of the Advances in Neural Information Processing Systems.

[32]

Yasasvi V. Peruvemba, Shubham Rai, Kapil Ahuja, and Akash Kumar. 2021. RL-Guided runtime-constrained heuristic exploration for logic synthesis. In Proceedings of the IEEE/ACM International Conference On Computer Aided Design. IEEE, 1–9.

Digital Library

[33]

Joachim Pistorius, Mike Hutton, Alan Mishchenko, and Robert Brayton. 2007. Benchmarking method and designs targeting logic synthesis for FPGAs. In Proceedings of the International Workshop on Logic and Synthesis. 230–237.

[34]

Yu Qian, Xuegong Zhou, Hao Zhou, and Lingli Wang. 2022. Efficient reinforcement learning framework for automated logic synthesis exploration. In Proceedings of the 2022 International Conference on Field-Programmable Technology. IEEE, 1–6.

[35]

Heinz Riener, Eleonora Testa, Winston Haaswijk, Alan Mishchenko, Luca Amarù, Giovanni De Micheli, and Mathias Soeken. 2019. Scalable generic logic synthesis: One approach to rule them all. In Proceedings of the 56th ACM/IEEE Design Automation Conference. ACM, 1–6.

Digital Library

[36]

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv:1707.06347. Retrieved from https://arxiv.org/abs/1707.06347

[37]

David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484–489.

[38]

Mathias Soeken. [n. d.]. CirKit. Retrieved from https://github.com/msoeken/cirkit. Accessed 2020.

[39]

Mathias Soeken, Heinz Riener, Winston Haaswijk, Eleonora Testa, Bruno Schmitt, Giulia Meuli, Fereshte Mozafari, and Giovanni De Micheli. 2018. The EPFL logic synthesis libraries. arXiv:1805.05121. Retrieved from https://arxiv.org/abs/1805.05121

[40]

Richard S. Sutton and Andrew G. Barto. 2005. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks 16, 1 (2005), 285–286.

Digital Library

[41]

Richard S. Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation. In Proceedings of the Advances in Neural Information Processing Systems. 1-7

[42]

Clifford Wolf. [n. d.]. PicoRV32 - A Size-optimized RISC-V CPU. Retrieved from https://github.com/YosysHQ/picorv32. Accessed 2020.

[43]

Nan Wu, Jiwon Lee, Yuan Xie, and Cong Hao. 2022. LOSTIN: Logic optimization via spatio-temporal information with hybrid graph models. In Proceedings of the 33rd IEEE International Conference on Application-specific Systems, Architectures and Processors.IEEE, 11–18.

[44]

Biying Xu, Yibo Lin, Xiyuan Tang, Shaolan Li, Linxiao Shen, Nan Sun, and David Z Pan. 2019. WellGAN: Generative-adversarial-network-guided well generation for analog/mixed-signal circuit layout. In Proceedings of the 56th Annual Design Automation Conference 2019. ACM, 1–6.

Digital Library

[45]

Chenghao Yang, Yinshui Xia, Zhufei Chu, and Xiaojing Zha. 2022. Logic synthesis optimization sequence tuning using RL-based LSTM and graph isomorphism network. IEEE Transactions on Circuits and Systems II: Express Briefs 69, 8 (2022), 3600–3604.

[46]

Saeyang Yang. 1991. Logic Synthesis and Optimization Benchmarks User Guide: Version 3.0. Citeseer.

[47]

Cunxi Yu. 2020. FlowTune: Practical multi-armed bandits in boolean optimization. In Proceedings of the IEEE/ACM International Conference On Computer Aided Design. IEEE, 1–9.

Digital Library

[48]

Cunxi Yu, Houping Xiao, and Giovanni De Micheli. 2018. Developing synthesis flows without human knowledge. In Proceedings of the 55th Annual Design Automation Conference. ACM, 1–6.

Digital Library

[49]

Cunxi Yu and Wang Zhou. 2020. Decision making in synthesis cross technologies using LSTMs and transfer learning. In Proceedings of the ACM/IEEE Workshop on Machine Learning for CAD. ACM, 55–60.

Digital Library

[50]

Yucheng Zhao, Guangting Wang, Chuanxin Tang, Chong Luo, Wenjun Zeng, and Zheng-Jun Zha. 2021. An empirical study of CNN, Transformer, and MLP. arXiv:2108.13002. Retrieved from https://arxiv.org/abs/2108.13002

[51]

Keren Zhu, Mingjie Liu, Hao Chen, Zheng Zhao, and David Z. Pan. 2020. Exploring logic optimizations with reinforcement learning and graph convolutional network. In Proceedings of the ACM/IEEE 2nd Workshop on Machine Learning for CAD. ACM, 145–150.

Digital Library

Cited By

Jiang HZou PLi XZhou ZZhao XZhang YGuo S(2024)DeLoSo: Detecting Logic Synthesis Optimization Faults Based on Configuration DiversityACM Transactions on Design Automation of Electronic Systems10.1145/370123230:1(1-26)Online publication date: 26-Oct-2024
https://dl.acm.org/doi/10.1145/3701232
Tseng I(2024)Pragmatic EDA Flow Runtime Prediction with Machine Learning and Automatic Outlier Removal2024 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS)10.1109/APCCAS62602.2024.10808773(203-207)Online publication date: 7-Nov-2024
https://doi.org/10.1109/APCCAS62602.2024.10808773

Index Terms

An Efficient Reinforcement Learning Based Framework for Exploring Logic Synthesis
1. Computing methodologies
  1. Machine learning
2. Hardware
  1. Electronic design automation
    1. Logic synthesis

Recommendations

Area-Driven FPGA Logic Synthesis Using Reinforcement Learning
ASPDAC '23: Proceedings of the 28th Asia and South Pacific Design Automation Conference

Logic synthesis involves a rich set of optimization algorithms applied in a specific sequence to a circuit netlist prior to technology mapping. A conventional approach is to apply a fixed "recipe" of such algorithms deemed to work well for a wide range ...
Exploring Logic Optimizations with Reinforcement Learning and Graph Convolutional Network
MLCAD '20: Proceedings of the 2020 ACM/IEEE Workshop on Machine Learning for CAD

Logic synthesis for combinational circuits is to find the minimum equivalent representation for Boolean logic functions. A well-adopted logic synthesis paradigm represents the Boolean logic with standardized logic networks, such as and-inverter graphs (...
Combinational logic synthesis for LUT based field programmable gate arrays

The increasing popularity of the field programmable gate-array (FPGA) technology has generated a great deal of interest in the algorithmic study and tool development for FPGA-specific design automation problems. The most widely used FPGAs are LUT based ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Design Automation of Electronic Systems

ACM Transactions on Design Automation of Electronic Systems Volume 29, Issue 2

March 2024

438 pages

EISSN:1557-7309

DOI:10.1145/3613564

Editor:
Jiang Hu
Texas A&M University, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 15 January 2024

Online AM: 10 November 2023

Accepted: 04 November 2023

Revised: 06 October 2023

Received: 16 March 2023

Published in TODAES Volume 29, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key R&D Program of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
928
Total Downloads

Downloads (Last 12 months)646
Downloads (Last 6 weeks)55

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang HZou PLi XZhou ZZhao XZhang YGuo S(2024)DeLoSo: Detecting Logic Synthesis Optimization Faults Based on Configuration DiversityACM Transactions on Design Automation of Electronic Systems10.1145/370123230:1(1-26)Online publication date: 26-Oct-2024
https://dl.acm.org/doi/10.1145/3701232
Tseng I(2024)Pragmatic EDA Flow Runtime Prediction with Machine Learning and Automatic Outlier Removal2024 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS)10.1109/APCCAS62602.2024.10808773(203-207)Online publication date: 7-Nov-2024
https://doi.org/10.1109/APCCAS62602.2024.10808773

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents