More Web Proxy on the site http://driver.im/

short-paper

TSTL: a language and tool for testing (demo)

Authors:

Pranjal MittalAuthors Info & Claims

ISSTA 2015: Proceedings of the 2015 International Symposium on Software Testing and Analysis

Pages 414 - 417

https://doi.org/10.1145/2771783.2784769

Published: 13 July 2015 Publication History

Abstract

Writing a test harness is a difficult and repetitive program- ming task, and the lack of tool support for customized auto- mated testing is an obstacle to the adoption of more sophis- ticated testing in industry. This paper presents TSTL, the Template Scripting Testing Language, which allows users to specify the general form of valid tests for a system in a simple but expressive language, and tools to support testing based on a TSTL definition. TSTL is a minimalist template- based domain-specific language, using the source language of the Software Under Test (SUT) to support most operations, but adding declarative idioms for testing. TSTL compiles to a common testing interface that hides the details of the SUT and provides support for logging, code coverage, delta debugging, and other core testing functionality, making it easy to write universal testing tools such as random testers or model checkers that apply to all TSTL-defined harnesses. TSTL is currently available for Python, but easily adapted to other languages as well.

References

[1]

J. Andrews, Y. R. Zhang, and A. Groce. Comparing automated unit testing strategies. Technical Report 736, Department of Computer Science, University of Western Ontario, December 2010.

[2]

H. Barringer and K. Havelund. TRACECONTRACT: a Scala DSL for trace analysis. In Formal Methods, pages 57–72, 2011.

Digital Library

[3]

D. Giannakopoulou, F. Howar, M. Isberner, T. Lauderdale, Z. Rakamari´ c, and V. Raman. Taming test inputs for separation assurance. In International Conference on Automated Software Engineering, pages 373–384, 2014.

Digital Library

[4]

M. Gligoric, T. Gvero, V. Jagannath, S. Khurshid, V. Kuncak, and D. Marinov. Test generation through programming in UDITA. In International Conference on Software Engineering, pages 225–234, 2010.

Digital Library

[5]

A. Groce, M. A. Alipour, C. Zhang, Y. Chen, and J. Regehr. Cause reduction for quick testing. In Software Testing, Verification and Validation (ICST), 2014 IEEE Seventh International Conference on, pages 243–252. IEEE, 2014.

Digital Library

[6]

A. Groce and M. Erwig. Finding common ground: choose, assert, and assume. In Workshop on Dynamic Analysis, pages 12–17, 2012.

Digital Library

[7]

A. Groce, K. Havelund, and M. Smith. From scripts to specifications: The evolution of a flight software testing effort. In International Conference on Software Engineering, pages 129–138, 2010.

Digital Library

[8]

A. Groce, G. Holzmann, and R. Joshi. Randomized differential testing as a prelude to formal verification. In International Conference on Software Engineering, pages 621–631, 2007.

Digital Library

[9]

A. Groce and J. Pinto. In NASA Formal Methods Symposium, 2015. To appear.

[10]

G. J. Holzmann. The SPIN Model Checker: Primer and Reference Manual. Addison-Wesley Professional, 2003.

Digital Library

[11]

D. Kroening, E. M. Clarke, and F. Lerda. A tool for checking ANSI-C programs. In Tools and Algorithms for the Construction and Analysis of Systems, pages 168–176, 2004.

[12]

W. McKeeman. Differential testing for software. Digital Technical Journal of Digital Equipment Corporation, 10(1):100–107, 1998.

[13]

A. Milicevic, S. Misailovic, D. Marinov, and S. Khurshid. Korat: A tool for generating structurally complex test inputs. In International Conference on Software Engineering, pages 771–774, 2007.

Digital Library

[14]

E. Shein. Python for beginners. Communications of the ACM, 58(3):19–21, 2015.

Digital Library

[15]

W. Visser, K. Havelund, G. Brat, S. Park, and F. Lerda. Model checking programs. Automated Software Engineering, 10(2):203–232, Apr. 2003.

Digital Library

[16]

X. Yang, Y. Chen, E. Eide, and J. Regehr. Finding and understanding bugs in C compilers. In ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 283–294, 2011.

Digital Library

[17]

A. Zeller and R. Hildebrandt. Simplifying and isolating failure-inducing input. Software Engineering, IEEE Transactions on, 28(2):183–200, 2002.

Digital Library

Cited By

Nilizadeh ALeavens GPăsăreanu CNoller Y(2023)JMLKelinci+: Detecting Semantic Bugs and Covering Branches with Valid Inputs Using Coverage-guided Fuzzing and Runtime Assertion CheckingFormal Aspects of Computing10.1145/360753836:1(1-24)Online publication date: 5-Aug-2023
https://dl.acm.org/doi/10.1145/3607538
Tse TLo DGorce APerscheid MHirschfeld RWong W(2023)Emerging Aspects of Software Fault LocalizationHandbook of Software Fault Localization10.1002/9781119880929.ch13(529-579)Online publication date: 21-Apr-2023
https://doi.org/10.1002/9781119880929.ch13
Nilizadeh ALeavens GPăsăreanu C(2021)Using a Guided Fuzzer and Preconditions to Achieve Branch Coverage with Valid InputsTests and Proofs10.1007/978-3-030-79379-1_5(72-84)Online publication date: 18-Jun-2021
https://doi.org/10.1007/978-3-030-79379-1_5
Show More Cited By

Index Terms

TSTL: a language and tool for testing (demo)
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

TSTL: the template scripting testing language

A test harness, in automated test generation, defines the set of valid tests for a system, as well as their correctness properties. The difficulty of writing test harnesses is a major obstacle to the adoption of automated test generation and model ...
Mutation Testing Cost Reduction Techniques: A Survey

Since the 1970s, researchers have widely used mutation as a testing technique, applying mainly it to validate test suites, as well as to validate test case strategies and test data generation. Mutation today is sufficiently mature for industrial ...
Grammar-driven generation of domain-specific language testing tools
OOPSLA '05: Companion to the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications

Domain-specific languages (DSLs) assist an end-user in writing a software program using idioms that are closer to the abstractions found in a specific problem domain. Language tool support for DSLs is lacking when compared to the capabilities provided ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2015: Proceedings of the 2015 International Symposium on Software Testing and Analysis

July 2015

447 pages

ISBN:9781450336208

DOI:10.1145/2771783

General Chair:
Michal Young
University of Oregon, USA
,
Program Chair:
Tao Xie
University of Illinois at Urbana-Champaign, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Science Foundation

Conference

ISSTA '15

Sponsor:

SIGSOFT

ISSTA '15: International Symposium on Software Testing and Analysis

July 13 - 17, 2015

MD, Baltimore, USA

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
178
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 22 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Nilizadeh ALeavens GPăsăreanu CNoller Y(2023)JMLKelinci+: Detecting Semantic Bugs and Covering Branches with Valid Inputs Using Coverage-guided Fuzzing and Runtime Assertion CheckingFormal Aspects of Computing10.1145/360753836:1(1-24)Online publication date: 5-Aug-2023
https://dl.acm.org/doi/10.1145/3607538
Tse TLo DGorce APerscheid MHirschfeld RWong W(2023)Emerging Aspects of Software Fault LocalizationHandbook of Software Fault Localization10.1002/9781119880929.ch13(529-579)Online publication date: 21-Apr-2023
https://doi.org/10.1002/9781119880929.ch13
Nilizadeh ALeavens GPăsăreanu C(2021)Using a Guided Fuzzer and Preconditions to Achieve Branch Coverage with Valid InputsTests and Proofs10.1007/978-3-030-79379-1_5(72-84)Online publication date: 18-Jun-2021
https://doi.org/10.1007/978-3-030-79379-1_5
Holmes JGroce A(2020)Using mutants to help developers distinguish and debug (compiler) faultsSoftware Testing, Verification and Reliability10.1002/stvr.172730:2Online publication date: 13-Jan-2020
https://doi.org/10.1002/stvr.1727
Groce AAhmed IJensen CMckenney PHolmes J(2018)How verified (or tested) is my code? Falsification-driven verification and testingAutomated Software Engineering10.5555/3288647.328871225:4(917-960)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.5555/3288647.3288712
Holmes JGroce A(2018)Causal Distance-Metric-Based Assistance for Debugging after Compiler Fuzzing2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE.2018.00027(166-177)Online publication date: Oct-2018
https://doi.org/10.1109/ISSRE.2018.00027
Groce AAhmed IJensen CMcKenney PHolmes J(2018)How verified (or tested) is my code? Falsification-driven verification and testingAutomated Software Engineering10.1007/s10515-018-0240-y25:4(917-960)Online publication date: 11-Jul-2018
https://doi.org/10.1007/s10515-018-0240-y
Holmes JGroce APinto JMittal PAzimi PKellar KO'brien J(2018)TSTLInternational Journal on Software Tools for Technology Transfer (STTT)10.1007/s10009-016-0445-y20:1(57-78)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1007/s10009-016-0445-y
Holmes JGroce ABultan TSen K(2017)A suite of tools for making effective use of automatically generated testsProceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3092703.3098220(356-359)Online publication date: 10-Jul-2017
https://dl.acm.org/doi/10.1145/3092703.3098220
Groce AHolmes JKellar KBultan TSen K(2017)One test to rule them allProceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3092703.3092704(1-11)Online publication date: 10-Jul-2017
https://dl.acm.org/doi/10.1145/3092703.3092704
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents