[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1109/ICSE48619.2023.00168acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Ex Pede Herculem: Augmenting Activity Transition Graph for Apps via Graph Convolution Network

Published: 26 July 2023 Publication History

Abstract

Mobile apps are indispensable for people's daily life. With the increase of GUI functions, apps have become more complex and diverse. As the Android app is event-driven, Activity Transition Graph (ATG) becomes an important way of app abstract and graphical user interface (GUI) modeling. Although existing works provide static and dynamic analysis to build ATG for applications, the completeness of ATG obtained is poor due to the low coverage of these techniques. To tackle this challenge, we propose a novel approach, ArchiDroid, to automatically augment the ATG via graph convolution network. It models both the semantics of activities and the graph structure of activity transitions to predict the transition between activities based on the seed ATG extracted by static analysis. The evaluation demonstrates that ArchiDroid can achieve 86% precision and 94% recall in predicting the transition between activities for augmenting ATG. We further apply the augmented ATG in two downstream tasks, i.e., guidance in automated GUI testing and assistance in app function design. Results show that the automated GUI testing tool integrated with ArchiDroid achieves 43% more activity coverage and detects 208% more bugs. Besides, ArchiDroid can predict the missing transition with 85% accuracy in real-world apps for assisting the app function design, and an interview case study further demonstrates its usefulness.

References

[1]
"Mobile internet use passes desktop for the first time." https://techcrunch.com/2016/11/01/, 2022.
[2]
"Android development," http://developer.android.com/reference/android, 2022.
[3]
L. Li, T. F. Bissyandé, M. Papadakis, S. Rasthofer, A. Bartel, D. Octeau, J. Klein, and Y. L. Traon, "Static analysis of android apps: A systematic literature review," Inf. Softw. Technol., 2017.
[4]
C. Liu and X. Xiao, "Promal: precise window transition graphs for android via synergy of program analysis and machine learning," in International Conference on Software Engineering. IEEE, 2022.
[5]
"Number of available applications in the google play store from december 2009 to june 2020," https://www.statista.com/statistics/266210/number-of-available-applications-in-the-google-play-store, 2022.
[6]
J. Gao, L. Li, T. F. Bissyandé, and J. Klein, "On the evolution of mobile app complexity," in 2019 24th international conference on engineering of complex computer systems (ICECCS). IEEE, 2019, pp. 200--209.
[7]
S. Chen, L. Fan, C. Chen, T. Su, W. Li, Y. Liu, and L. Xu, "Storydroid: Automated generation of storyboard for android apps," in ICSE. IEEE, 2019, pp. 596--607.
[8]
M. Fazzini, M. Prammer, M. d'Amorim, and A. Orso, "Automatically translating bug reports into test cases for mobile apps," in Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2018, pp. 141--152.
[9]
T. Cai, Z. Zhang, and P. Yang, "Fastbot: A multi-agent model-based test generation system beijing bytedance network technology co., ltd." in Proceedings of the IEEE/ACM 1st International Conference on Automation of Software Test, 2020, pp. 93--96.
[10]
Z. Liu, C. Chen, J. Wang, Y. Huang, J. Hu, and Q. Wang, "Nighthawk: Fully automated localizing ui display issues via visual understanding," 2022, pp. 1--16.
[11]
A. Noulas, S. Scellato, C. Mascolo, and M. Pontil, "An empirical study of geographic user activity patterns in foursquare," in AAAI, 2011.
[12]
T. Azim and I. Neamtiu, "Targeted and depth-first exploration for systematic testing of android apps," in OOPSLA, 2013.
[13]
Z. Liu, C. Chen, J. Wang, Y. Huang, J. Hu, and Q. Wang, "Owl eyes: Spotting ui display issues via visual understanding," in ASE. IEEE, 2020. [Online].
[14]
T. Gu, C. Sun, X. Ma, C. Cao, C. Xu, Y. Yao, Q. Zhang, J. Lu, and Z. Su, "Practical gui testing of android applications via model abstraction and refinement," in ICSE, 2019.
[15]
S. Feng and C. Chen, "Gifdroid: Automated replay of visual bug reports for android apps," in ICSE. ACM, 2022.
[16]
T. Su, G. Meng, Y. Chen, K. Wu, W. Yang, Y. Yao, G. Pu, Y. Liu, and Z. Su, "Guided, stochastic model-based gui testing of android apps," in Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, 2017, pp. 245--256.
[17]
S. Xi, S. Yang, X. Xiao, Y. Yao, Y. Xiong, F. Xu, H. Wang, P. Gao, Z. Liu, F. Xu, and J. Lu, "Deepintent: Deep icon-behavior learning for detecting intention-behavior discrepancy in mobile apps," in CCS. ACM, 2019.
[18]
D. Lai and J. Rubin, "Goal-driven exploration for android applications," in 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2019, pp. 115--127.
[19]
H. Zhang and A. Rountev, "Analysis and testing of notifications in android wear applications," in 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), 2017, pp. 347--357.
[20]
W. Guo, L. Shen, T. Su, X. Peng, and W. Xie, "Improving automated gui exploration of android apps via static dependency analysis," in 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2020, pp. 557--568.
[21]
K. Moran, C. Bernal-Cárdenas, M. Curcio, R. Bonett, and D. Poshyvanyk, "Machine learning-based prototyping of graphical user interfaces for mobile apps," TSE, 2018.
[22]
F. Behrang, S. P. Reiss, and A. Orso, "Guifetch: supporting app design and development through gui search," in Proceedings of the 5th International Conference on Mobile Software Engineering and Systems. ACM, 2018.
[23]
Z. Liu, C. Chen, J. Wang, and Q. Wang, "Guided bug crush: Assist manual gui testing of android apps via hint moves," in CHI 2022, 2022. [Online].
[24]
M. Wan, Y. Jin, D. Li, J. Gui, S. Mahajan, and W. G. J. Halfond, "Detecting display energy hotspots in android apps," Softw. Test. Verification Reliab., vol. 27, no. 6, 2017. [Online].
[25]
X. Xiao, X. Wang, Z. Cao, H. Wang, and P. Gao, "Iconintent: automatic identification of sensitive UI widgets based on icon classification for android apps," in ICSE. IEEE / ACM, 2019.
[26]
Y. Zhao, M. S. Laser, Y. Lyu, and N. Medvidovic, "Leveraging program analysis to reduce user-perceived latency in mobile applications," in ICSE, 2018.
[27]
H. Zhang, H. Wu, and A. Rountev, "Automated test generation for detection of leaks in android applications." Association for Computing Machinery, 2016.
[28]
T. Wu, X. Deng, J. Yan, and J. Zhang, "Analyses for specific defects in android applications: A survey," Frontiers of Computer Science, 2019.
[29]
S. Yang, H. Wu, H. Zhang, Y. Wang, C. Swaminathan, D. Yan, and A. Rountev, "Static window transition graphs for android," Automated Software Engineering, vol. 25, no. 4, pp. 833--873, 2018.
[30]
S. Yang, D. Yan, H. Wu, Y. Wang, and A. Rountev, "Static control-flow analysis of user-driven callbacks in android applications," in 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 1. IEEE, 2015, pp. 89--99.
[31]
D. Octeau, D. Luchaup, M. Dering, S. Jha, and P. McDaniel, "Composite constant propagation: Application to android inter-component communication analysis," in 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 1. IEEE, 2015, pp. 77--88.
[32]
Y. Zhang, Y. Sui, and J. Xue, "Launch-mode-aware context-sensitive activity transition analysis," in Proceedings of the 40th International Conference on Software Engineering, 2018, pp. 598--608.
[33]
W. Wang, W. Lam, and T. Xie, "An infrastructure approach to improving effectiveness of android ui testing tools," in ISSTA, 2021.
[34]
Y. Wang, H. Zhang, and A. Rountev, "On the unsoundness of static analysis for android guis," in Proceedings of the 5th ACM SIGPLAN International Workshop on State Of the Art in Program Analysis, 2016.
[35]
H. Cai and B. Ryder, "A longitudinal study of application structure and behaviors in android," TSE, 2020.
[36]
N. Mirzaei, J. Garcia, H. Bagheri, A. Sadeghi, and S. Malek, "Reducing combinatorics in gui testing of android applications," in ICSE, 2016.
[37]
W. Yang, M. R. Prasad, and T. Xie, "A grey-box approach for automated gui-model generation of mobile applications," in International Conference on Fundamental Approaches to Software Engineering. Springer, 2013, pp. 250--265.
[38]
A. Machiry, R. Tahiliani, and M. Naik, "Dynodroid: An input generation system for android apps," in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, 2013, pp. 224--234.
[39]
X. Zeng, D. Li, W. Zheng, F. Xia, Y. Deng, W. Lam, W. Yang, and T. Xie, "Automated test input generation for android: Are we really there yet in an industrial case?" in FSE, 2016.
[40]
K. Mao, M. Harman, and Y. Jia, "Sapienz: Multi-objective automated testing for android applications," in Proceedings of the 25th International Symposium on Software Testing and Analysis, 2016, pp. 94--105.
[41]
S. Van Der Burg and E. Dolstra, "Automating system tests using declarative virtual machines," in 2010 IEEE 21st International Symposium on Software Reliability Engineering. IEEE, 2010, pp. 181--190.
[42]
K. Wiklund, S. Eldh, D. Sundmark, and K. Lundqvist, "Impediments for software test automation: A systematic literature review," Software Testing, Verification and Reliability, vol. 27, no. 8, p. e1639, 2017.
[43]
Y. Li, Z. Yang, Y. Guo, and X. Chen, "Humanoid: a deep learning-based approach to automated black-box android app testing," in 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2019, pp. 1070--1073.
[44]
M. Pan, A. Huang, G. Wang, T. Zhang, and X. Li, "Reinforcement learning based curiosity-driven testing of android applications," in Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2020, pp. 153--164.
[45]
F. Behrang and A. Orso, "Test migration between mobile apps with similar functionality," in ASE, 2019.
[46]
"Ex pede herculem," https://en.wikipedia.org/wiki/Ex_pede_Herculem, 2022.
[47]
J. Yan, H. Liu, L. Pan, J. Yan, J. Zhang, and B. Liang, "Multiple-entry testing of android applications by constructing activity launching contexts," in 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 2020, pp. 457--468.
[48]
S. Arzt, S. Rasthofer, C. Fritz, E. Bodden, A. Bartel, J. Klein, Y. Le Traon, D. Octeau, and P. McDaniel, "Flowdroid: Precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps," Acm Sigplan Notices, vol. 49, no. 6, pp. 259--269, 2014.
[49]
"Apktool," https://ibotpeaches.github.io/Apktool/, 2022.
[50]
"pascal case," https://en.wikipedia.org/wiki/Camel_case, 2022.
[51]
B. Li, H. Zhou, J. He, M. Wang, Y. Yang, and L. Li, "On the sentence embeddings from pre-trained language models," EMNLP, 2020.
[52]
Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le, "Xlnet: Generalized autoregressive pretraining for language understanding," Advances in neural information processing systems, vol. 32, 2019.
[53]
P. Qin, X. Wang, W. Chen, C. Zhang, W. Xu, and W. Y. Wang, "Generative adversarial zero-shot relational learning for knowledge graphs," in AAAI, 2020.
[54]
W. Liu, P. Zhou, Z. Zhao, Z. Wang, Q. Ju, H. Deng, and P. Wang, "K-bert: Enabling language representation with knowledge graph," in AAAI, 2020.
[55]
T. N. Kipf and M. Welling, "Semi-supervised classification with graph convolutional networks," in 5th International Conference on Learning Representations, ICLR 2017, 2017.
[56]
Q. Li, Z. Han, and X.-M. Wu, "Deeper insights into graph convolutional networks for semi-supervised learning," in AAAI, 2018.
[57]
L. Yao, C. Mao, and Y. Luo, "Graph convolutional networks for text classification," in AAAI, 2019.
[58]
S. Zhang, H. Tong, J. Xu, and R. Maciejewski, "Graph convolutional networks: Algorithms, applications and open challenges," in CSoNet, 2018.
[59]
C. M. Bishop, Pattern recognition and machine learning. springer, 2006.
[60]
K. Kuznetsov, C. Fu, S. Gao, D. N. Jansen, L. Zhang, and A. Zeller, "Frontmatter: mining android user interfaces at scale," in FSE, 2021.
[61]
"F-droid," https://f-droid.org/, 2022.
[62]
Y. Li, Z. Yang, Y. Guo, and X. Chen, "Droidbot: a lightweight uiguided test input generator for android," in ICSE, 2017.
[63]
R. Tarjan, "Depth-first search and linear graph algorithms," SIAM journal on computing, 1972.
[64]
C. B. Seaman, "Qualitative methods in empirical studies of software engineering," IEEE Trans. Software Eng., 1999.
[65]
Y. Zhang, R. Jin, and Z. H. Zhou, "Understanding bag-of-words model: a statistical framework," IJMLC, 2010.
[66]
T. Mikolov, K. Chen, G. Corrado, and J. Dean, "Efficient estimation of word representations in vector space," Computer Science, 2013.
[67]
J. Pennington, R. Socher, and C. Manning, "Glove: Global vectors for word representation," in EMNLP, 2014.
[68]
G.-I. Brokos, P. Malakasiotis, and I. Androutsopoulos, "Using centroids of word embeddings and word mover's distance for biomedical document retrieval in question answering," arXiv preprint arXiv:1608.03905, 2016.
[69]
P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, "Enriching word vectors with subword information," TACL, 2017.
[70]
Q. Le and T. Mikolov, "Distributed representations of sentences and documents," in CIVR, 2014.
[71]
J. Sarzynska-Wawer, A. Wawer, A. Pawlak, J. Szymanowska, I. Stefaniak, M. Jarkiewicz, and L. Okruszek, "Detecting formal thought disorder by deep contextualized word representations," Psychiatry Research, vol. 304, p. 114135, 2021.
[72]
I. Staliunaite and I. Iacobacci, "Compositional and lexical semantics in roberta, BERT and distilbert: A case study on coqa," in EMNLP, 2020.
[73]
S. B. Kotsiantis, I. Zaharakis, and P. Pintelas, "Supervised machine learning: A review of classification techniques," Emerging artificial intelligence applications in computer engineering, 2007.
[74]
A. Berson, S. Smith, and K. Thearling, "An overview of data mining techniques," Building Data Mining Application for CRM, 2004.
[75]
L. Breiman, "Random forests," Machine learning, vol. 45, no. 1, pp. 5--32, 2001.
[76]
I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT press, 2016.
[77]
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," nature, 2015.
[78]
M. Al Hasan, V. Chaoji, S. Salem, and M. Zaki, "Link prediction using supervised learning," in SDM06: workshop on link analysis, counter-terrorism and security, vol. 30, 2006, pp. 798--805.
[79]
H. Wang, X. Shi, and D.-Y. Yeung, "Relational deep learning: A deep latent variable model for link prediction," in Thirty-first AAAI conference on artificial intelligence, 2017.
[80]
Y. Ma, Y. Huang, Z. Hu, X. Xiao, and X. Liu, "Paladin: Automated generation of reproducible test cases for android apps," in Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications, 2019, pp. 99--104.
[81]
S. Hao, B. Liu, S. Nath, W. G. Halfond, and R. Govindan, "Puma: Programmable ui-automation for large-scale dynamic analysis of mobile apps," in Proceedings of the 12th annual international conference on Mobile systems, applications, and services, 2014, pp. 204--217.
[82]
Y. Li, Z. Yang, Y. Guo, and X. Chen, "Droidbot: A lightweight uiguided test input generator for android," ser. ICSE-C '17, 2017.
[83]
S. Anand, M. Naik, M. J. Harrold, and H. Yang, "Automated concolic testing of smartphone apps," in Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering, 2012, pp. 1--11.
[84]
R. Mahmood, N. Mirzaei, and S. Malek, "Evodroid: Segmented evolutionary testing of android apps," in FSE, 2014.
[85]
S. R. Choudhary, A. Gorla, and A. Orso, "Automated test input generation for android: Are we there yet?(e)," in ASE, 2015.
[86]
W. Wang, D. Li, W. Yang, Y. Cao, Z. Zhang, Y. Deng, and T. Xie, "An empirical study of android test generation tools in industrial cases," in 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2018, pp. 738--748.
[87]
W. Choi, G. Necula, and K. Sen, "Guided gui testing of android apps with minimal restart and approximate learning," Acm Sigplan Notices, vol. 48, no. 10, pp. 623--640, 2013.
[88]
Z. Qin, Y. Tang, E. Novak, and Q. Li, "Mobiplay: A remote execution based record-and-replay tool for mobile applications," in ICSE, 2016.
[89]
W. Song, X. Qian, and J. Huang, "Ehbdroid: Beyond gui testing for android applications," in ASE, 2017.
[90]
L. Guo, R. Sharma, L. Yin, R. Lu, and K. Rong, "Automated competitor analysis using big data analytics: Evidence from the fitness mobile app business," Business Process Management Journal, 2017.
[91]
R. Fox, "Mobile app development: The effect of smartphones, mobile applications and geolocation services on the tourist experience," Ph.D. dissertation, University of Baltimore, 2017.
[92]
"Moneytracker app," https://github.com/yev-kanivets/open_money_tracker, 2022.
[93]
"Noisecapture," https://play.google.com/store/apps/details?id=org.noise_planet.noisecapture, 2022.
[94]
"Noisecapture user issue," https://github.com/Ifsttar/NoiseCapture/issues/114, 2022.
[95]
J. Brooke et al., "Sus-a quick and dirty usability scale," Usability evaluation in industry, 1996.
[96]
A. Narayanan, M. Chandramohan, R. Venkatesan, L. Chen, Y. Liu, and S. Jaiswal, "graph2vec: Learning distributed representations of graphs," CoRR, vol. abs/1707.05005, 2017. [Online]. Available: http://arxiv.org/abs/1707.05005
[97]
A. Rountev and D. Yan, "Static reference analysis for gui objects in android software," in Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization, 2014, pp. 143--153.
[98]
S. Feng and C. Chen, "Gifdroid: Automated replay of visual bug reports for android apps," International Conference on Software Engineering (ICSE), 2021.
[99]
K. Kuznetsov, C. Fu, S. Gao, D. N. Jansen, L. Zhang, and A. Zeller, "What do all these buttons do? statically mining android user interfaces at scale," 2021.
[100]
L. Lü and T. Zhou, "Link prediction in complex networks: A survey," Physica A: statistical mechanics and its applications, vol. 390, no. 6, pp. 1150--1170, 2011.
[101]
C. Shi, Y. Li, J. Zhang, Y. Sun, and S. Y. Philip, "A survey of heterogeneous information network analysis," IEEE Transactions on Knowledge and Data Engineering, 2016.
[102]
H. Chen, H. Yin, W. Wang, H. Wang, Q. V. H. Nguyen, and X. Li, "Pme: projected metric embedding on heterogeneous networks for link prediction," in KDD, 2018.
[103]
D. Liben-Nowell and J. Kleinberg, "The link-prediction problem for social networks," Journal of the American society for information science and technology, vol. 58, no. 7, pp. 1019--1031, 2007.
[104]
M. Schlichtkrull, T. N. Kipf, P. Bloem, R. v. d. Berg, I. Titov, and M. Welling, "Modeling relational data with graph convolutional networks," in European semantic web conference, 2018.
[105]
X. Li, N. Du, H. Li, K. Li, J. Gao, and A. Zhang, "A deep learning approach to link prediction in dynamic networks," in SDM, 2014.
[106]
M. Zhang and Y. Chen, "Link prediction based on graph neural networks," in NIPS, 2018.
[107]
S. M. Kazemi and D. Poole, "Simple embedding for link prediction in knowledge graphs," in NEURIPS, 2018.

Cited By

View all
  • (2025)Extraction and empirical evaluation of GUI-level invariants as GUI Oracles in mobile app testingInformation and Software Technology10.1016/j.infsof.2024.107531177:COnline publication date: 1-Jan-2025
  • (2024)Unblind Text Inputs: Predicting Hint-text of Text Input in Mobile Apps via LLMProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642939(1-20)Online publication date: 11-May-2024
  • (2024)Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware DecisionsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639180(1-13)Online publication date: 20-May-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
ICSE '23: Proceedings of the 45th International Conference on Software Engineering
May 2023
2713 pages
ISBN:9781665457019
  • General Chair:
  • John Grundy,
  • Program Co-chairs:
  • Lori Pollock,
  • Massimiliano Di Penta

Sponsors

In-Cooperation

  • IEEE CS

Publisher

IEEE Press

Publication History

Published: 26 July 2023

Check for updates

Author Tags

  1. GUI testing
  2. deep learning
  3. program analysis
  4. empirical study

Qualifiers

  • Research-article

Conference

ICSE '23
Sponsor:
ICSE '23: 45th International Conference on Software Engineering
May 14 - 20, 2023
Victoria, Melbourne, Australia

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)28
  • Downloads (Last 6 weeks)5
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Extraction and empirical evaluation of GUI-level invariants as GUI Oracles in mobile app testingInformation and Software Technology10.1016/j.infsof.2024.107531177:COnline publication date: 1-Jan-2025
  • (2024)Unblind Text Inputs: Predicting Hint-text of Text Input in Mobile Apps via LLMProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642939(1-20)Online publication date: 11-May-2024
  • (2024)Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware DecisionsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639180(1-13)Online publication date: 20-May-2024
  • (2024)Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language ModelProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639118(1-12)Online publication date: 20-May-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media