More Web Proxy on the site http://driver.im/

short-paper

Tool demo: testing configurable systems with FeatureIDE

Authors:

Mustafa Al-Hajjaji,

Sebastian Krieter,

Reimar Schröter,

Gunter SaakeAuthors Info & Claims

GPCE 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences

Pages 173 - 177

https://doi.org/10.1145/2993236.2993254

Published: 20 October 2016 Publication History

Abstract

Most software systems are designed to provide custom functionality using configuration options. Testing such systems is challenging as running tests of a single configuration is often not sufficient, because defects may appear in other configurations. Ideally, all configurations of a software system should be tested, which is usually not applicable in practice due to the combinatorial explosion with respect to the configuration options. Multiple sampling strategies aim to reduce the set of tested configurations to a feasible amount, such as T-wise sampling, random configurations, and user-defined configurations. However, these strategies are often not applied in practice as they require manual effort or a specialized testing framework. Within our tool FeatureIDE, we integrate all aforementioned strategies and reduce the manual effort by automating the process of generating and testing configurations. Furthermore, we provide support for unit testing to avoid redundant test executions and for variability-aware testing. With this extension of FeatureIDE, we aim to make recent testing techniques for configurable systems applicable in practice.

References

[1]

M. Al-Hajjaji, S. Krieter, T. Thüm, M. Lochau, and G. Saake. IncLing: Efficient Product-Line Testing Using Incremental Pairwise Sampling. In GPCE, 2016. To appear.

Digital Library

[2]

M. Al-Hajjaji, T. Thüm, J. Meinicke, M. Lochau, and G. Saake. Similarity-Based Prioritization in Software Product-Line Testing. In SPLC, pp. 197–206. ACM, 2014.

Digital Library

[3]

S. Apel, D. Batory, C. Kästner, and G. Saake. Feature-Oriented Software Product Lines: Concepts and Implementation. Springer, 2013.

Digital Library

[4]

S. Apel, C. Kästner, and C. Lengauer. Language-Independent and Automated Software Composition: The FeatureHouse Experience. TSE, 39(1):63–79, 2013.

Digital Library

[5]

S. Apel, A. von Rhein, P. Wendler, A. Größlinger, and D. Beyer. Strategies for Product-Line Verification: Case Studies and Experiments. In ICSE, pp. 482–491. IEEE, 2013.

Digital Library

[6]

D. Batory. Feature Models, Grammars, and Propositional Formulas. In SPLC, pp. 7–20. Springer, 2005.

Digital Library

[7]

D. Batory, J. N. Sarvela, and A. Rauschmayer. Scaling Step-Wise Refinement. TSE, 30(6):355–371, 2004.

Digital Library

[8]

B. Beckert, R. Hähnle, and P. Schmitt. Verification of Object-Oriented Software: The KeY Approach. Springer, 2007.

Digital Library

[9]

D. Benavides, S. Segura, and A. Ruiz-Cortés. Automated Analysis of Feature Models 20 Years Later: A Literature Review. Information Systems, 35(6):615–708, 2010.

Digital Library

[10]

J. Bürdek, M. Lochau, S. Bauregger, A. Holzer, A. von Rhein, S. Apel, and D. Beyer. Facilitating Reuse in Multi-goal Test-Suite Generation for Software Product Lines. In FASE, pp. 84–99. Springer, 2015.

[11]

V. Chvatal. A Greedy Heuristic for the Set-Covering Problem. MOR, 4(3):233–235, 1979.

Digital Library

[12]

B. J. Garvin, M. B. Cohen, and M. B. Dwyer. Evaluating Improvements to a Meta-Heuristic Search for Constrained Interaction Testing. EMSE, 16(1):61–102, 2011.

Digital Library

[13]

K. Havelund and T. Pressburger. Model Checking Java Programs Using Java PathFinder. STTT, 2(4):366–381, 2000.

[14]

C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. L. Traon. PLEDGE: A Product Line Editor and Test Generation Tool. In SPLC, pp. 126–129. ACM, 2013.

Digital Library

[15]

K. J. Hoffman, P. Eugster, and S. Jagannathan. Semanticsaware Trace Analysis. In PLDI, pp. 453–464. ACM, 2009.

Digital Library

[16]

M. Jackson and P. Zave. Distributed Feature Composition: A Virtual Architecture for Telecommunications Services. TSE, 24(10):831–847, 1998.

Digital Library

[17]

M. F. Johansen, Ø. Haugen, and F. Fleurey. Properties of Realistic Feature Models Make Combinatorial Testing of Product Lines Feasible. In MODELS, pp. 638–652. Springer, 2011.

Digital Library

[18]

M. F. Johansen, Ø. Haugen, and F. Fleurey. An Algorithm for Generating T-Wise Covering Arrays from Large Feature Models. In SPLC, pp. 46–55. ACM, 2012.

Digital Library

[19]

G. Kiczales, J. Lamping, A. Mendhekar, C. Maeda, C. Lopes, J.-M. Loingtier, and J. Irwin. Aspect-Oriented Programming. In ECOOP, pp. 220–242. Springer, 1997.

[20]

C. H. P. Kim, D. Marinov, S. Khurshid, D. Batory, S. Souto, P. Barros, and M. d’Amorim. SPLat: Lightweight Dynamic Analysis for Reducing Combinatorics in Testing Configurable Systems. In ESEC/FSE, pp. 257–267. ACM, 2013.

Digital Library

[21]

D. R. Kuhn, D. R. Wallace, and A. M. Gallo Jr. Software Fault Interactions and Implications for Software Testing. TSE, 30(6):418–421, 2004.

Digital Library

[22]

D. Le Berre and A. Parrain. The sat4j Library, Release 2.2, System Description. JSAT, 7:59–64, 2010.

[23]

J. McGregor. Testing a Software Product Line. In Testing Techniques in Software Engineering, pp. 104–140. Springer, 2010.

[24]

F. Medeiros, C. Kästner, M. Ribeiro, S. Nadi, and R. Gheyi. The Love/Hate Relationship with the C Preprocessor: An Interview Study. In ECOOP, pp. 495–518. Schloss Dagstuhl– Leibniz-Zentrum fuer Informatik, 2015.

[25]

J. Meinicke, T. Thüm, R. Schröter, S. Krieter, F. Benduhn, G. Saake, and T. Leich. FeatureIDE: Taming the Preprocessor Wilderness. In ICSE, pp. 629–632. ACM, 2016.

Digital Library

[26]

J. Meinicke, C.-P. Wong, C. Kästner, T. Thüm, and G. Saake. On Essential Configuration Complexity: Measuring Interactions in Highly-configurable Systems. In ASE, pp. 483–494. ACM, 2016.

Digital Library

[27]

J. A. Pereira, S. Krieter, J. Meinicke, R. Schröter, G. Saake, and T. Leich. FeatureIDE: Scalable Product Configuration of Variable Systems. In ICSR, pp. 397–401. Springer, 2016.

Digital Library

[28]

G. Perrouin, S. Sen, J. Klein, B. Baudry, and Y. Le Traon. Automated and Scalable T-Wise Test Case Generation Strategies for Software Product Lines. In ICST, pp. 459–468. IEEE, 2010.

Digital Library

[29]

M. Steffens, S. Oster, M. Lochau, and T. Fogdal. Industrial Evaluation of Pairwise SPL Testing with MoSo-PoLiTe. In VaMoS, pp. 55–62. ACM, 2012.

Digital Library

[30]

R. Tartler, D. Lohmann, J. Sincero, and W. Schröder-Preikschat. Feature Consistency in Compile-Time-Configurable System Software: Facing the Linux 10,000 Feature Problem. In EuroSys, pp. 47–60. ACM, 2011.

Digital Library

[31]

T. Thüm, S. Apel, C. Kästner, I. Schaefer, and G. Saake. A Classification and Survey of Analysis Strategies for Software Product Lines. CSUR, 47(1):6:1–6:45, 2014.

Digital Library

[32]

T. Thüm, C. Kästner, F. Benduhn, J. Meinicke, G. Saake, and T. Leich. FeatureIDE: An Extensible Framework for Feature-Oriented Software Development. SCP, 79(0):70–85, 2014.

Digital Library

[33]

T. Thüm, J. Meinicke, F. Benduhn, M. Hentschel, A. von Rhein, and G. Saake. Potential Synergies of Theorem Proving and Model Checking for Software Product Lines. In SPLC, pp. 177–186. ACM, 2014.

Digital Library

[34]

A. von Rhein, S. Apel, and F. Raimondi. Introducing Binary Decision Diagrams in the Explicit-State Verification of Java code. In JavaPathfinder Workshop, 2011.

[35]

B. Xin, W. N. Sumner, and X. Zhang. Efficient Program Execution Indexing. In PLDI, pp. 238–248. ACM, 2008.

Digital Library

Cited By

Mendonça WVergilio SMichelon GEgyed AAssunção WFelfernig AFuentes L(2022)Test2FeatureProceedings of the 26th ACM International Systems and Software Product Line Conference - Volume B10.1145/3503229.3547031(62-65)Online publication date: 12-Sep-2022
https://dl.acm.org/doi/10.1145/3503229.3547031
Pett TKrieter SRunge TThüm TLochau MSchaefer I(2021)Stability of Product-Line Samplingin Continuous IntegrationProceedings of the 15th International Working Conference on Variability Modelling of Software-Intensive Systems10.1145/3442391.3442410(1-9)Online publication date: 9-Feb-2021
https://dl.acm.org/doi/10.1145/3442391.3442410
Ferreira FDiniz JSilva CFigueiredo EWeyns DPerrouin GWeyns D(2019)Testing Tools for Configurable Software SystemsProceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems10.1145/3302333.3302344(1-10)Online publication date: 6-Feb-2019
https://dl.acm.org/doi/10.1145/3302333.3302344
Show More Cited By

Index Terms

Tool demo: testing configurable systems with FeatureIDE
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools

Recommendations

Tool demo: testing configurable systems with FeatureIDE
GPCE '16

Most software systems are designed to provide custom functionality using configuration options. Testing such systems is challenging as running tests of a single configuration is often not sufficient, because defects may appear in other configurations. ...
Automating Test Reuse for Highly Configurable Software
SPLC '19: Proceedings of the 23rd International Systems and Software Product Line Conference - Volume A

Dealing with highly configurable systems is generally very complex. Hundreds of different analysis techniques have been conceived to deal with different aspects of configurable systems. One large focal point is the testing of configurable software. This ...
Automated test reuse for highly configurable software
Abstract
Dealing with highly configurable systems is generally very complex. Researchers and practitioners have conceived hundreds of different analysis techniques to deal with different aspects of configurable systems. One large focal point is the testing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GPCE 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences

October 2016

212 pages

ISBN:9781450344463

DOI:10.1145/2993236

General Chair:
Bernd Fischer
Stellenbosch University, South Africa
,
Program Chair:
Ina Schaefer
TU Braunschweig, Germany

ACM SIGPLAN Notices Volume 52, Issue 3
GPCE '16
March 2017
212 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3093335
Editor:
Matthew Fluet
Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

GPCE '16

Sponsor:

SIGPLAN

GPCE '16: Generative Programming: Concepts and Experiences

October 31 - November 1, 2016

Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 56 of 180 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
167
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mendonça WVergilio SMichelon GEgyed AAssunção WFelfernig AFuentes L(2022)Test2FeatureProceedings of the 26th ACM International Systems and Software Product Line Conference - Volume B10.1145/3503229.3547031(62-65)Online publication date: 12-Sep-2022
https://dl.acm.org/doi/10.1145/3503229.3547031
Pett TKrieter SRunge TThüm TLochau MSchaefer I(2021)Stability of Product-Line Samplingin Continuous IntegrationProceedings of the 15th International Working Conference on Variability Modelling of Software-Intensive Systems10.1145/3442391.3442410(1-9)Online publication date: 9-Feb-2021
https://dl.acm.org/doi/10.1145/3442391.3442410
Ferreira FDiniz JSilva CFigueiredo EWeyns DPerrouin GWeyns D(2019)Testing Tools for Configurable Software SystemsProceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems10.1145/3302333.3302344(1-10)Online publication date: 6-Feb-2019
https://dl.acm.org/doi/10.1145/3302333.3302344
Al-Hajjaji MThüm TLochau MMeinicke JSaake G(2019)Effective product-line testing using similarity-based product prioritizationSoftware and Systems Modeling (SoSyM)10.1007/s10270-016-0569-218:1(499-521)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1007/s10270-016-0569-2
Luthmann LGerecht TLochau M(2019)Sampling strategies for product lines with unbounded parametric real-time constraintsInternational Journal on Software Tools for Technology Transfer10.1007/s10009-019-00532-421:6(613-633)Online publication date: 24-Sep-2019
https://doi.org/10.1007/s10009-019-00532-4
Thüm THoorn AApel SBürdek JGetir SHeinrich RJung RKowal MLochau MSchaefer IWalter J(2019)Performance Analysis Strategies for Software Variants and VersionsManaged Software Evolution10.1007/978-3-030-13499-0_8(175-206)Online publication date: 27-Jun-2019
https://doi.org/10.1007/978-3-030-13499-0_8
Mansoor NSaddler JSilva BBagheri HCohen MFarritor SLeavens GGarcia APăsăreanu C(2018)Modeling and testing a family of surgical robots: an experience reportProceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3236024.3275534(785-790)Online publication date: 26-Oct-2018
https://dl.acm.org/doi/10.1145/3236024.3275534
Varshosaz MAl-Hajjaji MThüm TRunge TMousavi MSchaefer I(2018)A classification of product sampling for software product linesProceedings of the 22nd International Systems and Software Product Line Conference - Volume 110.1145/3233027.3233035(1-13)Online publication date: 10-Sep-2018
https://dl.acm.org/doi/10.1145/3233027.3233035
Krieter SThüm TSchulze SSchröter RSaake GChaudron MCrnkovic IChechik MHarman M(2018)Propagating configuration decisions with modal implication graphsProceedings of the 40th International Conference on Software Engineering10.1145/3180155.3180159(898-909)Online publication date: 27-May-2018
https://dl.acm.org/doi/10.1145/3180155.3180159
Krieter SPinnecke MKrüger JSprey JSontag CThüm TLeich TSaake G(2017)FeatureIDEProceedings of the 21st International Systems and Software Product Line Conference - Volume B10.1145/3109729.3109751(42-45)Online publication date: 25-Sep-2017
https://dl.acm.org/doi/10.1145/3109729.3109751
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents