More Web Proxy on the site http://driver.im/

research-article

A framework for automated testing of javascript web applications

Authors:

Simon Holm Jensen,

Anders Møller,

Frank TipAuthors Info & Claims

ICSE '11: Proceedings of the 33rd International Conference on Software Engineering

Pages 571 - 580

https://doi.org/10.1145/1985793.1985871

Published: 21 May 2011 Publication History

Abstract

Current practice in testing JavaScript web applications requires manual construction of test cases, which is difficult and tedious. We present a framework for feedback-directed automated test generation for JavaScript in which execution is monitored to collect information that directs the test generator towards inputs that yield increased coverage. We implemented several instantiations of the framework, corresponding to variations on feedback-directed random testing, in a tool called Artemis. Experiments on a suite of JavaScript applications demonstrate that a simple instantiation of the framework that uses event handler registrations as feedback information produces surprisingly good coverage if enough tests are generated. By also using coverage information and read-write sets as feedback information, a slightly better level of coverage can be achieved, and sometimes with many fewer tests. The generated tests can be used for detecting HTML validity problems and other programming errors.

References

[1]

S. Artzi, J. Dolby, F. Tip, and M. Pistoia. Practical fault localization for dynamic web applications. In Proc. 32nd Int. Conf. on Software Engineering, ICSE '10, May 2010.

Digital Library

[2]

S. Artzi, A. Kiezun, J. Dolby, F. Tip, D. Dig, A. M. Paradkar, and M. D. Ernst. Finding bugs in dynamic web applications. In Proc. Int. Symp. on Software Testing and Analysis, ISSTA '08, July 2008.

Digital Library

[3]

P. Boonstoppel, C. Cadar, and D. R. Engler. RWset: Attacking path explosion in constraint-based test generation. In Proc. 14th Int. Conf. on Tools and Algorithms for the Construction and Analysis of Systems, TACAS '08, March-April 2008.

Digital Library

[4]

R. C. Bryce and A. M. Memon. Test suite prioritization by interaction coverage. In Proc. Workshop on Domain Specific Approaches to Software Test Automation, DOSTA '07, September 2007.

Digital Library

[5]

C. Duda, G. Frey, D. Kossmann, R. Matter, and C. Zhou. AJAX crawl: Making AJAX applications searchable. In Proc. 25th Int. Conf. on Data Engineering, ICDE '09, March-April 2009.

Digital Library

[6]

ECMA. ECMAScript Language Specification, 3rd edition. ECMA-262.

[7]

P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In Proc. ACM SIGPLAN Conf. on Programming Language Design and Implementation, PLDI '05, June 2005.

Digital Library

[8]

S. Guarnieri and B. Livshits. Gatekeeper: Mostly static enforcement of security and reliability policies for JavaScript code. In Proc. 18th USENIX Security Symposium, August 2009.

Digital Library

[9]

A. Guha, S. Krishnamurthi, and T. Jim. Using static analysis for Ajax intrusion detection. In Proc. 18th Int. Conf. on World Wide Web, WWW '09, April 2009.

Digital Library

[10]

P. Heidegger and P. Thiemann. JSConTest: Contract-driven testing of JavaScript code. In Proc. 48th Int. Conf. on Objects, Components, Models and Patterns, TOOLS '10, LNCS. Springer, June-July 2010.

Digital Library

[11]

H.-Y. Hsu and A. Orso. MINTS: A general framework and tool for supporting test-suite minimization. In Proc. 31st Int. Conf. on Software Engineering, ICSE '09, May 2009.

Digital Library

[12]

S. H. Jensen, A. Møller, and P. Thiemann. Type analysis for JavaScript. In Proc. 16th Int. Static Analysis Symposium, SAS '09, volume 5673 of LNCS. Springer, August 2009.

Digital Library

[13]

A. Le Hors et al. Document Object Model (DOM) level 3 core specification, April 2004. W3C Recommendation. http://www.w3.org/TR/DOM-Level-3-Core/.

[14]

A. Marchetto and P. Tonella. Search-based testing of Ajax web applications. In Proc. 1st Int. Symp. on Search Based Software Engineering, SSBSE '09, May 2009.

Digital Library

[15]

A. Marchetto, P. Tonella, and F. Ricca. State-based testing of Ajax web applications. In Proc. 1st Int. Conf. on Software Testing, Verification, and Validation, ICST '08, April 2008.

Digital Library

[16]

A. M. Memon. An event-flow model of GUI-based applications for testing. Software Testing, Verification & Reliability, 17(3):137--157, 2007.

Digital Library

[17]

A. Mesbah, E. Bozdag, and A. van Deursen. Crawling AJAX by inferring user interface state changes. In Proc. 8th Int. Conf. on Web Engineering, ICWE '08, July 2008.

Digital Library

[18]

A. Mesbah and A. van Deursen. Invariant-based automatic testing of AJAX user interfaces. In Proc. 31st Int. Conf. on Software Engineering, ICSE '09, May 2009.

Digital Library

[19]

C. Pacheco, S. K. Lahiri, M. D. Ernst, and T. Ball. Feedback-directed random test generation. In Proc. 29th Int. Conf. on Software Engineering, ICSE '07, May 2007.

Digital Library

[20]

G. Richards, S. Lebresne, B. Burg, and J. Vitek. An analysis of the dynamic behavior of JavaScript programs. In Proc. ACM SIGPLAN Conf. on Programming Language Design and Implementation, PLDI '10, June 2010.

Digital Library

[21]

G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. Prioritizing test cases for regression testing. IEEE Trans. on Software Engineering, 27(10):929--948, 2001.

Digital Library

[22]

P. Saxena, D. Akhawe, S. Hanna, S. McCamant, D. Song, and F. Mao. A symbolic execution framework for JavaScript. In Proc. 31st IEEE Symp. on Security and Privacy, S&P '10, May 2010.

Digital Library

[23]

D. Schepers et al. Document Object Model (DOM) level 3 events specification, September 2009. W3C Working Draft. http://www.w3.org/TR/DOM-Level-3-Events/.

[24]

K. Sen, D. Marinov, and G. Agha. CUTE: a concolic unit testing engine for C. In Proc. 10th European Software Engineering Conf. / 13th ACM SIGSOFT Int. Symp. on Foundations of Software Engineering, ESEC/FSE '05, September 2005.

Digital Library

[25]

X. Yuan and A. M. Memon. Generating event sequence-based test cases using GUI runtime state feedback. IEEE Trans. on Software Engineering, 36(1):81--95, 2010.

Digital Library

[26]

X. Yuan and A. M. Memon. Iterative execution-feedback model-directed GUI testing. Information & Software Technology, 52(5):559--575, 2010.

Digital Library

Cited By

Olsthoorn MStallenberg DPanichella A(2024)Syntest-JavaScript: Automated Unit-Level Test Case Generation for JavaScriptProceedings of the 17th ACM/IEEE International Workshop on Search-Based and Fuzz Testing10.1145/3643659.3643928(21-24)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3643659.3643928
Clerissi DDenaro GMobilio MMariani L(2024)DBInputs: Exploiting Persistent Data to Improve Automated GUI TestingIEEE Transactions on Software Engineering10.1109/TSE.2024.343900250:9(2412-2436)Online publication date: 6-Aug-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3439002
Clerissi DDenaro GMobilio MMariani L(2024)Guess the State: Exploiting Determinism to Improve GUI Exploration EfficiencyIEEE Transactions on Software Engineering10.1109/TSE.2024.336658650:4(836-853)Online publication date: 16-Feb-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3366586
Show More Cited By

Index Terms

A framework for automated testing of javascript web applications
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Automated Acceptance Testing of JavaScript Web Applications
WCRE '12: Proceedings of the 2012 19th Working Conference on Reverse Engineering

Acceptance testing is an important part of software development and it is performed to ensure that a system delivers its required functionalities. Today, most modern interactive web applications are designed using Web 2.0 technologies, many among them ...
Mutode: generic JavaScript and Node.js mutation testing tool
ISSTA 2018: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis

Mutation testing is a technique in which faults (mutants) are injected into a program or application to assess its test suite effectiveness. It works by inserting mutants and running the application’s test suite to identify if the mutants are detected (...
Automated user interface testing for web applications and TestComplete
CUBE '12: Proceedings of the CUBE International Information Technology Conference

Automated software testing is an important and time-saving activity used by software testing teams, working on rapid and large scale software projects. TestComplete is an example of a currently widely used testing tool. However, its test recorder tool ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '11: Proceedings of the 33rd International Conference on Software Engineering

May 2011

1258 pages

ISBN:9781450304450

DOI:10.1145/1985793

General Chair:
Richard N. Taylor
UC Irvine, USA
,
Program Chairs:
Harald Gall
University of Zurich, Switzerland
,
Nenad Medvidović
University of Southern California, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 May 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE11

Sponsor:

SIGSOFT

ICSE11: International Conference on Software Engineering

May 21 - 28, 2011

HI, Waikiki, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

144
Total Citations
View Citations
1,836
Total Downloads

Downloads (Last 12 months)89
Downloads (Last 6 weeks)17

Reflects downloads up to 16 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Olsthoorn MStallenberg DPanichella A(2024)Syntest-JavaScript: Automated Unit-Level Test Case Generation for JavaScriptProceedings of the 17th ACM/IEEE International Workshop on Search-Based and Fuzz Testing10.1145/3643659.3643928(21-24)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3643659.3643928
Clerissi DDenaro GMobilio MMariani L(2024)DBInputs: Exploiting Persistent Data to Improve Automated GUI TestingIEEE Transactions on Software Engineering10.1109/TSE.2024.343900250:9(2412-2436)Online publication date: 6-Aug-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3439002
Clerissi DDenaro GMobilio MMariani L(2024)Guess the State: Exploiting Determinism to Improve GUI Exploration EfficiencyIEEE Transactions on Software Engineering10.1109/TSE.2024.336658650:4(836-853)Online publication date: 16-Feb-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3366586
Schäfer MNadi SEghbali ATip F(2024)An Empirical Evaluation of Using Large Language Models for Automated Unit Test GenerationIEEE Transactions on Software Engineering10.1109/TSE.2023.333495550:1(85-105)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TSE.2023.3334955
Huo JZhang CFu ZLu C(2024)JavaScript-Based Power Line Communication Signal-Simulation2024 IEEE 15th International Conference on Software Engineering and Service Science (ICSESS)10.1109/ICSESS62520.2024.10719053(143-147)Online publication date: 13-Sep-2024
https://doi.org/10.1109/ICSESS62520.2024.10719053
Ganji MAlimadadi STip FChandra SBlincoe KTonella P(2023)Code Coverage Criteria for Asynchronous ProgramsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616292(1307-1319)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616292
Wang LHuang QShao WWang JLiu XWang F(2023)Felix: A Model of Detecting Off-chain Abnormal States in Decentralized Applications2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom)10.1109/TrustCom60117.2023.00056(275-283)Online publication date: 1-Nov-2023
https://doi.org/10.1109/TrustCom60117.2023.00056
Xia XFeng Y(2023)DyFuzz: Skeleton-based Fuzzing for Python Libraries2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security (QRS)10.1109/QRS60937.2023.00040(325-336)Online publication date: 22-Oct-2023
https://doi.org/10.1109/QRS60937.2023.00040
Yan YZheng YLiu XMedvidovic NWang W(2023)Adhere: Automated Detection and Repair of Intrusive Ads2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00051(486-498)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00051
Kim IWang WKwon YZhang X(2023)BFTDETECTOR: Automatic Detection of Business Flow Tampering for Digital Content Service2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00048(448-459)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00048
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents