[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/2509136.2509552acmconferencesArticle/Chapter ViewAbstractPublication PagessplashConference Proceedingsconference-collections
research-article

Guided GUI testing of android apps with minimal restart and approximate learning

Published: 29 October 2013 Publication History

Abstract

Smartphones and tablets with rich graphical user interfaces (GUI) are becoming increasingly popular. Hundreds of thousands of specialized applications, called apps, are available for such mobile platforms. Manual testing is the most popular technique for testing graphical user interfaces of such apps. Manual testing is often tedious and error-prone. In this paper, we propose an automated technique, called Swift-Hand, for generating sequences of test inputs for Android apps. The technique uses machine learning to learn a model of the app during testing, uses the learned model to generate user inputs that visit unexplored states of the app, and uses the execution of the app on the generated inputs to refine the model. A key feature of the testing algorithm is that it avoids restarting the app, which is a significantly more expensive operation than executing the app on a sequence of inputs. An important insight behind our testing algorithm is that we do not need to learn a precise model of an app, which is often computationally intensive, if our goal is to simply guide test execution into unexplored parts of the state space. We have implemented our testing algorithm in a publicly available tool for Android apps written in Java. Our experimental results show that we can achieve significantly better coverage than traditional random testing and L*-based testing in a given time budget. Our algorithm also reaches peak coverage faster than both random and L*-based testing.

References

[1]
Managing the Activity Lifecycle. http://developer.android.com/training/basics/activity-lifecycle/index.html.
[2]
MonkeyRunner. http://developer.android.com/tools/help/monkeyrunner_concepts.html.
[3]
UI/Application Exerciser Monkey. http://developer.android.com/tools/help/monkey.html.
[4]
axml, read and write Android binary xml files. http://code.google.com/p/axml/, 2012.
[5]
R. Alur, P. Cerny, P. Madhusudan, and W. Nam. Synthesis of interface specifications for java classes. In POPL, pages 98--109, 2005.
[6]
D. Amalfitano, A. R. Fasolino, P. Tramontana, S. D. Carmine, and A. M. Memon. Using GUI ripping for automated testing of Android applications. In ASE, pages 258--261, 2012.
[7]
S. Anand, M. Naik, M. J. Harrold, and H. Yang. Automated concolic testing of smartphone apps. In SIGSOFT FSE, page 59, 2012.
[8]
A. A. Andrews, J. Offutt, and R. T. Alexander. Testing web applications by modeling with FSMs. Software and System Modeling, 4(3):326--345, 2005.
[9]
D. Angluin. Inference of reversible languages. J. ACM, 29(3):741--765, July 1982.
[10]
D. Angluin. Learning regular sets from queries and counterex-amples. Inf. Comput., 75(2):87--106, 1987.
[11]
F. Belli. Finite-state testing and analysis of graphical user interfaces. In 12th International Symposium on Software Reliability Engineering (ISSRE'01), page 34. IEEE Computer Society, 2001.
[12]
T. Berg, O. Grinchtein, B. Jonsson, M. Leucker, H. Raffelt, and B. Steffen. On the correspondence between conformance testing and regular inference. In FASE, pages 175--189, 2005.
[13]
J. M. Cobleigh, D. Giannakopoulou, and C. S. Pasareanu. Learning assumptions for compositional verification. In TACAS, pages 331--346, 2003.
[14]
F. Coste, D. Fredouille, C. Kermorvant, and C. de la Higuera. Introducing domain and typing bias in automata inference. In ICGI, pages 115--126, 2004.
[15]
J. N. Departarnento and P. Garcia. Identifying regular languages in polynomial. In Advances in Structural and Syntactic Pattern Recognition, volume 5 of Series in Machine Perception and Artificial Intelligence, pages 99--108. World Scientific, 1992.
[16]
P. Godefroid, N. Klarlund, and K. Sen. DART: directed automated random testing. In PLDI, pages 213--223, 2005.
[17]
A. Groce, D. Peled, and M. Yannakakis. AMC: An adaptive model checker. In CAV, pages 521--525, 2002.
[18]
A. Groce, D. Peled, and M. Yannakakis. Adaptive model checking. Logic Journal of the IGPL, 14(5):729--744, 2006.
[19]
R. Groz, M.-N. Irfan, and C. Oriat. Algorithmic improvements on regular inference of software models and perspectives for security testing. In ISoLA (1), pages 444--457, 2012.
[20]
C. Hu and I. Neamtiu. A GUI bug finding framework for Android applications. In SAC, pages 1490--1491, 2011.
[21]
B. Lambeau, C. Damas, and P. Dupont. State-merging DFA induction algorithms with mandatory merge constraints. In ICGI, pages 139--153, 2008.
[22]
K. Lang, B. Pearlmutter, and R. Price. Results of the Abbadingo One DFA learning competition and a new evidence-driven state merging algorithm, 1998.
[23]
K. J. Lang. Random DFA's can be approximately learned from sparse uniform examples. In COLT, pages 45--52, 1992.
[24]
X. Leroy. Java bytecode verification: algorithms and formalizations. Journal of Automated Reasoning, 30(3--4):235--269, 2003.
[25]
A. MacHiry, R. Tahiliani, and M. Naik. Dynodroid: An input generation system for Android apps. In SIGSOFT FSE, pages 224--235, 2013.
[26]
K. Meinke and M. A. Sindhu. Incremental learning-based testing for reactive systems. In TAP, pages 134--151, 2011.
[27]
K. Meinke and N. Walkinshaw. Model-based testing and model inference. In ISoLA (1), pages 440--443, 2012.
[28]
A. M. Memon. An event-flow model of GUI-based applications for testing. Softw. Test., Verif. Reliab., 17(3):137--157, 2007.
[29]
A. Mesbah, A. van Deursen, and S. Lenselink. Crawling Ajax-based web applications through dynamic analysis of user interface state changes. TWEB, 6(1):3, 2012.
[30]
N. Mirzaei, S. Malek, C. S. Pasareanu, N. Esfahani, and R. Mahmood. Testing Android apps through symbolic execution. ACM SIGSOFT Software Engineering Notes, 37(6):1--5, 2012.
[31]
J. Nevo and P. Crégut. ASMDEX. http://asm.ow2.org/asmdex-index.html, 2012.
[32]
W. M. Newman. A system for interactive graphical programming. In Proc. of the spring joint computer conference (AFIPS'68 (Spring)), pages 47--54. ACM, 1968.
[33]
C. D. Nguyen, A. Marchetto, and P. Tonella. Combining model-based and combinatorial testing for effective test case generation. In ISSTA, pages 100--110, 2012.
[34]
D. Peled, M. Y. Vardi, and M. Yannakakis. Black box checking. In FORTE, pages 225--240, 1999.
[35]
H. Raffelt, B. Steffen, and T. Margaria. Dynamic testing via automata learning. In Haifa Verification Conference, pages 136--152, 2007.
[36]
V. Rastogi, Y. Chen, and W. Enck. Appsplayground: automatic security analysis of smartphone applications. In CODASPY, pages 209--220, 2013.
[37]
H. Reza, S. Endapally, and E. Grant. A model-based approach for testing GUI using hierarchical predicate transition nets. In International Conference on Information Technology (ITNG'07), pages 366--370. IEEE Computer Society, 2007.
[38]
R. L. Rivest and R. E. Schapire. Inference of finite automata using homing sequences (extended abstract). In STOC, pages 411--420, 1989.
[39]
R. K. Shehady and D. P. Siewiorek. A methodology to automate user interface testing using variable finite state machines. In FTCS, pages 80--88, 1997.
[40]
T. Takala, M. Katara, and J. Harty. Experiences of system- level model-based GUI testing of an Android application. In ICST, pages 377--386, 2011.
[41]
N. Walkinshaw, K. Bogdanov, J. Derrick, and J. Paris. Increasing functional coverage by inductive testing: A case study. In ICTSS, pages 126--141, 2010.
[42]
N. Walkinshaw, J. Derrick, and Q. Guo. Iterative refinement of reverse-engineered models by model-based testing. In FM, pages 305--320, 2009.
[43]
L. White and H. Almezen. Generating test cases for GUI responsibilities using complete interaction sequences. In 11th International Symposium on Software Reliability Engineering (ISSRE'00), page 110. IEEE Computer Society, 2000.
[44]
T. Xie and D. Notkin. Mutually enhancing test generation and specification inference. In FATES, pages 60--69, 2003.
[45]
W. Yang, M. R. Prasad, and T. Xie. A grey-box approach for automated GUI-model generation of mobile applications. In FASE, pages 250--265, 2013.
[46]
X. Yuan, M. Cohen, and A. M. Memon. Covering array sampling of input event sequences for automated GUI testing. In ASE '07: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, pages 405--408, New York, NY, USA, 2007. ACM.
[47]
X. Yuan and A. M. Memon. Iterative execution-feedback model-directed GUI testing. Information & Software Technology, 52(5):559--575, 2010.

Cited By

View all
  • (2024)Navigating Mobile Testing Evaluation: A Comprehensive Statistical Analysis of Android GUI Testing MetricsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695476(944-956)Online publication date: 27-Oct-2024
  • (2024)DinoDroid: Testing Android Apps Using Deep Q-NetworksACM Transactions on Software Engineering and Methodology10.1145/365215033:5(1-24)Online publication date: 4-Jun-2024
  • (2024)GuiEvo: Automated Evolution of Mobile App UIsProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644936(335-347)Online publication date: 15-Apr-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
OOPSLA '13: Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications
October 2013
904 pages
ISBN:9781450323741
DOI:10.1145/2509136
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. android
  2. automata
  3. gui testing
  4. learning

Qualifiers

  • Research-article

Conference

SPLASH '13
Sponsor:

Acceptance Rates

OOPSLA '13 Paper Acceptance Rate 50 of 189 submissions, 26%;
Overall Acceptance Rate 268 of 1,244 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)84
  • Downloads (Last 6 weeks)9
Reflects downloads up to 11 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Navigating Mobile Testing Evaluation: A Comprehensive Statistical Analysis of Android GUI Testing MetricsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695476(944-956)Online publication date: 27-Oct-2024
  • (2024)DinoDroid: Testing Android Apps Using Deep Q-NetworksACM Transactions on Software Engineering and Methodology10.1145/365215033:5(1-24)Online publication date: 4-Jun-2024
  • (2024)GuiEvo: Automated Evolution of Mobile App UIsProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644936(335-347)Online publication date: 15-Apr-2024
  • (2024)A reinforcement learning-based approach to testing GUI of moblie applicationsWorld Wide Web10.1007/s11280-024-01252-927:2Online publication date: 22-Feb-2024
  • (2023)Automata-Based Trace Analysis for Aiding Diagnosing GUI Testing Tools for AndroidProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616361(592-604)Online publication date: 30-Nov-2023
  • (2023)Model-based Testing for a Family of Mobile ApplicationsProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume A10.1145/3579027.3608996(242-253)Online publication date: 28-Aug-2023
  • (2023)Route: Roads Not Taken in UI TestingACM Transactions on Software Engineering and Methodology10.1145/357185132:3(1-25)Online publication date: 26-Apr-2023
  • (2023)Enhancing Mobile App Bug Reporting via Real-Time Understanding of Reproduction StepsIEEE Transactions on Software Engineering10.1109/TSE.2022.317402849:3(1246-1272)Online publication date: 1-Mar-2023
  • (2023)Columbus: Android App Testing through Systematic Callback ExplorationProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00121(1381-1392)Online publication date: 14-May-2023
  • (2023)Badge: Prioritizing UI Events with Hierarchical Multi-Armed Bandits for Automated UI Testing2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00083(894-905)Online publication date: May-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media