More Web Proxy on the site http://driver.im/

research-article

On the correctness of GPU programs

Author:

Chao PengAuthors Info & Claims

ISSTA 2019: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

Pages 443 - 447

https://doi.org/10.1145/3293882.3338989

Published: 10 July 2019 Publication History

Abstract

Testing is an important and challenging part of software development and its effectiveness depends on the quality of test cases. However, there exists no means of measuring quality of tests developed for GPU programs and as a result, no test case generation techniques for GPU programs aiming at high test effectiveness. Existing criteria for sequential and multithreaded CPU programs cannot be directly applied to GPU programs as GPU follows a completely different memory and execution model.

We surveyed existing work on GPU program verification and bug fixes of open source GPU programs. Based on our findings, we define barrier, branch and loop coverage criteria and propose a set of mutation operators to measure fault finding capabilities of test cases. CLTestCheck, a framework for measuring quality of tests developed for GPU programs by code coverage analysis, fault seeding and work-group schedule amplification has been developed and evaluated using industry standard benchmarks. Experiments show that the framework is able to automatically measure test effectiveness and reveal unusual behaviours. Our planned work includes data flow coverage adopted for GPU programs to probe the underlying cause of unusual kernel behaviours and a more comprehensive work-group scheduler. We also plan to design and develop an automatic test case generator aiming at generating high quality test suites for GPU programs.

References

[1]

2017. CUDA Zone. https://developer.nvidia.com/cuda-zone 2018. clang: a C language family frontend for LLVM. http://clang.llvm.org/

[2]

Adam Betts, Nathan Chong, Alastair Donaldson, Shaz Qadeer, and Paul Thomson. 2012. GPUVerify: a verifier for GPU kernels. In ACM SIGPLAN Notices, Vol. 47. ACM, 113–132.

Digital Library

[3]

Arkady Bron, Eitan Farchi, Yonit Magid, Yarden Nir, and Shmuel Ur. 2005. Applications of synchronization coverage. In Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming. ACM, 206–212.

Digital Library

[4]

Xia Cai and Michael R Lyu. 2005. The effect of code coverage on fault detection under different testing profiles. ACM SIGSOFT Software Engineering Notes 30, 4 (2005), 1–7.

Digital Library

[5]

Shuai Che, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W Sheaffer, Sang-Ha Lee, and Kevin Skadron. 2009. Rodinia: A benchmark suite for heterogeneous computing. In 2009 IEEE International Symposium on Workload Characterization (IISWC). IEEE, 44–54.

Digital Library

[6]

Peter Collingbourne, Cristian Cadar, and Paul HJ Kelly. 2011. Symbolic testing of OpenCL code. In Haifa Verification Conference. Springer, 203–218.

Digital Library

[7]

Gregory Gay, Ajitha Rajan, Matt Staats, Michael Whalen, and Mats PE Heimdahl. 2016. The effect of program and model structure on the effectiveness of mc/dc test adequacy coverage. ACM Transactions on Software Engineering and Methodology (TOSEM) 25, 3 (2016), 25.

Digital Library

[8]

Khronos OpenCL Working Group. 2017. The OpenCL Specification Version 2.2.

[9]

Shin Hong, Jaemin Ahn, Sangmin Park, Moonzoo Kim, and Mary Jean Harrold. 2012. Testing concurrent programs to achieve high synchronization coverage. In Proceedings of the 2012 International Symposium on Software Testing and Analysis. ACM, 210–220.

Digital Library

[10]

Alan Leung, Manish Gupta, Yuvraj Agarwal, Rajesh Gupta, Ranjit Jhala, and Sorin Lerner. 2012. Verifying GPU kernels by test amplification. In ACM SIGPLAN Notices, Vol. 47. ACM, 383–394.

Digital Library

[11]

Guodong Li and Ganesh Gopalakrishnan. 2010. Scalable SMT-based verification of GPU kernel functions. In Proceedings of the eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 187–196.

Digital Library

[12]

Guodong Li, Peng Li, Geof Sawaya, Ganesh Gopalakrishnan, Indradeep Ghosh, and Sreeranga P Rajan. 2012. GKLEE: concolic verification and test generation for GPUs. In ACM SIGPLAN Notices, Vol. 47. ACM, 215–224.

Digital Library

[13]

Phil McMinn. 2011. Search-based software testing: Past, present and future. In 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops. IEEE, 153–163.

Digital Library

[14]

Akbar Siami Namin and James H Andrews. 2009. The influence of size and coverage on test suite effectiveness. In Proceedings of the Eighteenth International Symposium on Software Testing and Analysis. ACM, 57–68.

Digital Library

[15]

Chao Peng and Ajitha Rajan. 2019. CLTestCheck: Measuring Test Effectiveness for GPU Kernels. In International Conference on Fundamental Approaches to Software Engineering. Springer, 315–331.

[16]

LN Pouchet and T Yuki. 2015. PolyBench/C 4.1. Retrieved May2015 from http://web.cse.ohio-state.edu/ pouchet/software/polybench (2015).

[17]

Ajitha Rajan and Mats PE Heimdahl. 2009. Coverage metrics for requirementsbased testing. University of Minnesota.

[18]

Ajitha Rajan, Subodh Sharma, Peter Schrammel, and Daniel Kroening. 2014. Accelerated test execution using gpus. In Proceedings of the 29th ACM/IEEE international conference on Automated software engineering. ACM, 97–102.

Digital Library

[19]

Shubhabrata Sengupta, Mark Harris, Yao Zhang, and John D Owens. 2007. Scan primitives for GPU computing. In Graphics hardware, Vol. 2007. 97–106.

Digital Library

[20]

Tyler Sorensen and Alastair F Donaldson. 2016. The Hitchhiker’s Guide to Cross-Platform OpenCL Application Development. In Proceedings of the 4th International Workshop on OpenCL. ACM, 2.

Digital Library

[21]

John A Stratton, Christopher Rodrigues, I-Jui Sung, Nady Obeid, Li-Wen Chang, Nasser Anssari, Geng Daniel Liu, and Wen-mei W Hwu. 2012. Parboil: A revised benchmark suite for scientific and commercial throughput computing. Center for Reliable and High-Performance Computing 127 (2012).

[22]

Michal Young. 2008. Software testing and analysis: process, principles, and techniques. John Wiley & Sons.

Digital Library

[23]

Mai Zheng, Vignesh T Ravi, Feng Qin, and Gagan Agrawal. 2011. GRace: a low-overhead mechanism for detecting data races in GPU programs. In ACM SIGPLAN Notices, Vol. 46. ACM, 135–146.

Digital Library

Cited By

Pujol RJorba JTabani HKosmidis LMezzetti EAbella JCazorla F(2023)Vector Extensions in COTS Processors to Increase Guaranteed Performance in Real-Time SystemsACM Transactions on Embedded Computing Systems10.1145/356105422:2(1-26)Online publication date: 24-Jan-2023
https://dl.acm.org/doi/10.1145/3561054

Index Terms

On the correctness of GPU programs
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Automated test generation for OpenCL kernels using fuzzing and constraint solving
GPGPU '20: Proceedings of the 13th Annual Workshop on General Purpose Processing using Graphics Processing Unit

Graphics Processing Units (GPUs) are massively parallel processors offering performance acceleration and energy efficiency unmatched by current processors (CPUs) in computers. These advantages along with recent advances in the programmability of GPUs ...
On the Efficacy of a Fused CPU+GPU Processor (or APU) for Parallel Computing
SAAHPC '11: Proceedings of the 2011 Symposium on Application Accelerators in High-Performance Computing

The graphics processing unit (GPU) has made significant strides as an accelerator in parallel computing. However, because the GPU has resided out on PCIe as a discrete device, the performance of GPU applications can be bottlenecked by data transfers ...
Automatic and Portable Mapping of Data Parallel Programs to OpenCL for GPU-Based Heterogeneous Systems

General-purpose GPU-based systems are highly attractive, as they give potentially massive performance at little cost. Realizing such potential is challenging due to the complexity of programming. This article presents a compiler-based approach to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2019: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

July 2019

451 pages

ISBN:9781450362245

DOI:10.1145/3293882

General Chair:
Dongmei Zhang
Microsoft Research, China
,
Program Chair:
Anders Møller
Aarhus University, Denmark

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '19

Sponsor:

SIGSOFT

ISSTA '19: 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

July 15 - 19, 2019

Beijing, China

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
196
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pujol RJorba JTabani HKosmidis LMezzetti EAbella JCazorla F(2023)Vector Extensions in COTS Processors to Increase Guaranteed Performance in Real-Time SystemsACM Transactions on Embedded Computing Systems10.1145/356105422:2(1-26)Online publication date: 24-Jan-2023
https://dl.acm.org/doi/10.1145/3561054

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents