Software based tools for simulation are not keeping up with the demands for increased chip and system design complexity. In this paper, we describe a cycle-accurate and cycle-reproducible large-scale FPGA platform that is designed from the ground up to accelerate logic verification of the Bluegene/Q compute node ASIC, a multi-processor SOC implemented in IBM's 45 nm SOI CMOS technology. This paper discusses the challenges for constructing such large-scale FPGA platforms, including design partitioning, clocking & synchronization, and debugging support, as well as our approach for addressing these challenges without sacrificing cycle accuracy and cycle reproducibility. The resulting fullchip simulation of the Bluegene/Q compute node ASIC runs at a simulated processor clock speed of 4 MHz, over 100,000 times faster than the logic level software simulation of the same design. The vast increase in simulation speed provides a new capability in the design cycle that proved to be instrumental in logic verification as well as early software development and performance validation for Bluegene/Q.

References

[1]

P. Wang, J. Collins, C. Weaver, B. Kuttanna, S. S. G. Chinya, E. Schuchman, O. Schilling, T. Doil, S. Steibl, and H. Wang, "Intel Atom processor core made FPGA Synthesizable," in Proceedings of the 17th Annual ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA'09), ACM, 2009.

Digital Library

Google Scholar

[2]

G. Schelle, J. Collins, E. Schuchman, P. Wang, X. Zou, G. Chinya, R. Plate, T. Mattner, F. Olbrich, P. Hammarlund, R. Singhal, J. Brayton, S. Steibl, and H. Wang, "Intel Nehalem Processor Core Made FPGA Synthesizable," in Proceedings of the 18th Annual ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA'10), ACM, 2010, Monterey, California.

Digital Library

Google Scholar

[3]

M. Gschwind, V. Salapura, and D. Maurer, "FPGA Prototyping of a RISC Processor Core for Embedded Applications," IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 9, no. 2, pp. 241--250, 2001.

Digital Library

Google Scholar

[4]

J. Ributzka, Y. Hayashi, F. Chen, and G. R. Gao, "DEEP: an iterative FPGA-based many-core emulation system for chip verification and architecture research," in Proceedings of the 19th ACM/SIGDA international symposium on Field programmable gate arrays, pp. 115--118, ACM, 2011.

Digital Library

Google Scholar

[5]

P. Subramanian, J. Patil, and M. K. Saxena, "FPGA prototyping of a multi-million gate System-on-Chip (SoC) design for wireless USB applications," in Proceedings of the 2009 International Conference on Wireless Communications and Mobile Computing: Connecting the World Wirelessly, pp. 1355--1358, ACM, 2009.

Digital Library

Google Scholar

[6]

J. Wawrzynek, D. Patterson, M. Oskin, S.-L. Lu, C. Kozyrakis, J. Hoe, D. Chiou, and K. Asanovic, "RAMP: Research Accelerator for Multiple Processors," IEEE Micro, vol. 27, no. 2, pp. 46--57, 2007.

Digital Library

Google Scholar

[7]

A. Krasnov, A. Schultz, J. Wawrzynek, G. Gibeling, and P.-Y. Droz, "RAMP Blue: A Message-Passing Manycore System in FPGAs," in International Conference on Field Programmable Logic and Applications, 2007, pp. 54--61, 2007.

Crossref

Google Scholar

[8]

C. Chang, K. Kuusilinna, B. Richards, A. Chen, R. Brodersen, and B.Nikolic, "Rapid design and analysis of communication systems using the BEE hardware emulation environment," in Proceedings of the 14th IEEE International Workshop on Rapid Systems Prototyping, 2003, pp. 148--154, IEEE, 2003.

Digital Library

Google Scholar

[9]

C.-Y. Huang, Y.-F. Yin, C.-J. Hsu, T. B. Huang, and T.-M. Chang, "SoC HW/SW verification and validation," in Proceedings of the 16th Asia and South Pacific Design Automation Conference, pp. 297--300, IEEE, 2011.

Digital Library

Google Scholar

[10]

R. Haring, "The Blue Gene/Q Compute Chip." http://www.hotchips.org/archives/hc23/HC23-papers/HC23.18.1-manycore/HC23.18.121.BlueGene-IBM_BQC_HC23_20110818.pdf, 2011.

Google Scholar

[11]

Xilinx, "Virtex-5 FPGA Configuration User Guide." http://www.xilinx.com/support/documentation/user_guides/ug191.pdf, 2011.

Google Scholar

[12]

Lawrence Livermore National Laboratory, "ASC Sequoia Benchmark Codes." https://asc.llnl.gov/sequoia/benchmarks/, 2009.

Google Scholar

Cited By

View all

Zang XLuo QShao ZZhang JYoung EWong M(2024)Dynamic Multi-FPGA Prototyping Platforms with Simultaneous Networking, Placement and RoutingProceedings of the Great Lakes Symposium on VLSI 202410.1145/3649476.3658713(433-439)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3649476.3658713
Whangbo JLim EZhang CAnderson KGonzalez AGupta RKrishnakumar NKarandikar SNikolić BShao YAsanović K(2024)FireAxe: Partitioned FPGA-Accelerated Simulation of Large-Scale RTL Designs2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA59077.2024.00044(501-515)Online publication date: 29-Jun-2024
https://doi.org/10.1109/ISCA59077.2024.00044
Chirkov GWentzlaff DAamodt TJerger NSwift M(2023)SMAPPIC: Scalable Multi-FPGA Architecture Prototype Platform in the CloudProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3575693.3575753(733-746)Online publication date: 27-Jan-2023
https://dl.acm.org/doi/10.1145/3575693.3575753
Show More Cited By

Index Terms

A cycle-accurate, cycle-reproducible multi-FPGA system for accelerating multi-core processor simulation
1. Computer systems organization
2. Hardware
  1. Hardware validation

Recommendations

A Reconfigurable Processor Architecture Combining Multi-core and Reconfigurable Processing Unit
CIT '10: Proceedings of the 2010 10th IEEE International Conference on Computer and Information Technology

It’s a promising way to improve performance significantly by adding reconfigurable processing unit to a general purpose processor. In this paper, a Reconfigurable Multi-Core (RMC) architecture combining general multi-core and reconfigurable logic is ...
Design and implementation of a plesiochronous multi-core 4x4 network-on-chip FPGA platform with MPI HAL support
FPGAworld '09: Proceedings of the 6th FPGAworld Conference

The Multi-Core NoC is a 4 by 4 Mesh NoC targeted for Altera FPGAs. It implements a deflective routing policy and is used to connect sixteen NIOS II processors. Each NIOS II is connected to the NoC via an address-mapped Resource Network Interface.

The ...
A reconfigurable processor architecture combining multi-core and reconfigurable processing units

It's a promising way to improve performance significantly by adding reconfigurable processing unit (RPU) to a general purpose processor. In this paper, a Reconfigurable Multi-Core (RMC) architecture combining general multi-core and reconfigurable logic ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

FPGA '12: Proceedings of the ACM/SIGDA international symposium on Field Programmable Gate Arrays

February 2012

352 pages

ISBN:9781450311557

DOI:10.1145/2145694

General Chair:
Katherine Compton
University of Wisconsin-Madison
,
Program Chair:
Brad Hutchings
Brigham Young University

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 February 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

FPGA '12

Sponsor:

SIGDA

FPGA '12: ACM/SIGDA International Symposium on Field Programmable Gate Arrays

February 22 - 24, 2012

California, Monterey, USA

Acceptance Rates

FPGA '12 Paper Acceptance Rate 20 of 87 submissions, 23%;

Overall Acceptance Rate 125 of 627 submissions, 20%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

79
Total Citations
View Citations
862
Total Downloads

Downloads (Last 12 months)39
Downloads (Last 6 weeks)9

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zang XLuo QShao ZZhang JYoung EWong M(2024)Dynamic Multi-FPGA Prototyping Platforms with Simultaneous Networking, Placement and RoutingProceedings of the Great Lakes Symposium on VLSI 202410.1145/3649476.3658713(433-439)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3649476.3658713
Whangbo JLim EZhang CAnderson KGonzalez AGupta RKrishnakumar NKarandikar SNikolić BShao YAsanović K(2024)FireAxe: Partitioned FPGA-Accelerated Simulation of Large-Scale RTL Designs2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA59077.2024.00044(501-515)Online publication date: 29-Jun-2024
https://doi.org/10.1109/ISCA59077.2024.00044
Chirkov GWentzlaff DAamodt TJerger NSwift M(2023)SMAPPIC: Scalable Multi-FPGA Architecture Prototype Platform in the CloudProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3575693.3575753(733-746)Online publication date: 27-Jan-2023
https://dl.acm.org/doi/10.1145/3575693.3575753
Attia SBetz V(2023)Toward Software-like Debugging for FPGAs via Checkpointing and Transaction-based Co-SimulationACM Transactions on Reconfigurable Technology and Systems10.1145/355252116:2(1-24)Online publication date: 10-May-2023
https://dl.acm.org/doi/10.1145/3552521
Chen YChang YZhang KChen MBao Y(2023)REMU: Enabling Cost-Effective Checkpointing and Deterministic Replay in FPGA-based Emulation2023 IEEE 41st International Conference on Computer Design (ICCD)10.1109/ICCD58817.2023.00014(21-29)Online publication date: 6-Nov-2023
https://doi.org/10.1109/ICCD58817.2023.00014
Farooq UBaig IBhatti MMehrez HKumar AGupta M(2023)Prototyping using multi-FPGA platformMicroprocessors & Microsystems10.1016/j.micpro.2022.10475196:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.micpro.2022.104751
Lan MHuang LYang LMa SYan RWang YXu W(2022)Late-Stage Optimization of Modern ILP Processor Cores via FPGA SimulationApplied Sciences10.3390/app12231222512:23(12225)Online publication date: 29-Nov-2022
https://doi.org/10.3390/app122312225
Huang PGuo LSun LZhang X(2022)A Two-Stage Method for Routing in Field-Programmable Gate Arrays with Time-Division MultiplexingTsinghua Science and Technology10.26599/TST.2021.901009227:6(902-911)Online publication date: Dec-2022
https://doi.org/10.26599/TST.2021.9010092
Herbst SRutsch GEcker WHorowitz M(2022)An Open-Source Framework for FPGA Emulation of Analog/Mixed-Signal Integrated Circuit DesignsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2021.310251641:7(2223-2236)Online publication date: Jul-2022
https://doi.org/10.1109/TCAD.2021.3102516
Feng HLi WWang SZhou JPang YTian CZhang Y(2022)A Framework of Embedded Logic Analyzer for FPGA2022 IEEE 5th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC)10.1109/IMCEC55388.2022.10019825(609-615)Online publication date: 16-Dec-2022
https://doi.org/10.1109/IMCEC55388.2022.10019825
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A Reconfigurable Processor Architecture Combining Multi-core and Reconfigurable Processing Unit

Design and implementation of a plesiochronous multi-core 4x4 network-on-chip FPGA platform with MPI HAL support

A reconfigurable processor architecture combining multi-core and reconfigurable processing units