[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3624032.3624040acmotherconferencesArticle/Chapter ViewAbstractPublication PagessastConference Proceedingsconference-collections
research-article

Automating Android Rotation Vector Testing in Google's Compatibility Test Suite Using a Robotic Arm

Published: 17 October 2023 Publication History

Abstract

Software testing is one of the essential phases of software development. In this context, test automation has gained significant traction for meeting immediate requirements while upholding result quality. While both automated and manual methods are employed for software testing, manual approaches are susceptible to inaccuracies and human errors despite the benefits of automation. This research is focused on Google’s Android OS, pivotal for a variety of global mobile devices, which must adhere to specific quality requirements. Our study focuses on the Compatibility Test Suite tool, specifically the Rotation Vector Computer Vision Crosscheck (RV) Test case designed for evaluating smartphone sensor capabilities. We introduce a three-axis robotic arm designed to automate the RV test execution, thereby minimizing operational failures and expediting Android smartphones’ quality testing processes. We compared RV test execution in a real-world company using our automated solution against human testers. The proposed robotic arm demonstrated a 75% accuracy rate, surpassing the 37% accuracy achieved by human testers. This significant disparity underscores the potential of our automation approach to mitigate manual errors while ensuring robust and effective testing processes.

Supplementary Material

Presentation video (sast2023-8-video.zip)

References

[1]
Daniel Asfaw. 2015. Benefits of automated testing over manual testing. International Journal of Innovative Research in Information Security 2, 1 (2015), 5–13.
[2]
Debdeep Banerjee and Kevin Yu. 2018. Robotic Arm-Based Face Recognition Software Test Automation. IEEE Access 6 (2018), 37858–37868. https://doi.org/10.1109/ACCESS.2018.2854754
[3]
Debdeep Banerjee, Kevin Yu, and Garima Aggarwal. 2018. Image Rectification Software Test Automation Using a Robotic ARM. IEEE Access 6 (2018), 34075–34085. https://doi.org/10.1109/ACCESS.2018.2846761
[4]
Stefan Berner, Roland Weber, and Rudolf K Keller. 2005. Observations and lessons learned from automated testing. In Proceedings of the 27th international conference on Software engineering. 571–579.
[5]
SM Bindu Bhargavi and V Suma. 2022. A Survey of the Software Test Methods and Identification of Critical Success Factors for Automation. SN Computer Science 3, 6 (2022), 449.
[6]
Android Developers. 2023. Compatibility Test Suite. url = https://source.android.com/compatibility/cts. "[Accessed: April/2023]".
[7]
Android Developers. 2023. Compatibility Test Suite Verifier. url = https://source.android.com/docs/compatibility/cts/verifier. "[Accessed: April/2023]".
[8]
Android Developers. 2023. Rotation Computer Vision Crosscheck Vector. url = https://source.android.com/compatibility/cts/rotation-vector. "[Accessed: April/2023]".
[9]
Demian Frister, Andreas Oberweis, and Aleksandar Goranov. 2020. Automated Testing of Mobile Applications Using a Robotic Arm. In 2020 International Conference on Computational Science and Computational Intelligence (CSCI). 1729–1735. https://doi.org/10.1109/CSCI51800.2020.00321
[10]
Heidilyn Veloso Gamido and Marlon Viray Gamido. 2019. Comparative review of the features of automated software testing tools. International Journal of Electrical and Computer Engineering 9, 5 (2019), 4473.
[11]
Klaus Haller. 2013. Mobile testing. ACM SIGSOFT Software Engineering Notes 38, 6 (2013), 1–8.
[12]
Mona Erfani Joorabchi, Ali Mesbah, and Philippe Kruchten. 2013. Real challenges in mobile app development. In 2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE, 15–24.
[13]
Wazir Zada Khan, Yang Xiang, Mohammed Y Aalsalem, and Quratulain Arshad. 2012. Mobile phone sensing systems: A survey. IEEE Communications Surveys & Tutorials 15, 1 (2012), 402–427.
[14]
Divya Kumar and Krishn Kumar Mishra. 2016. The impacts of test automation on software’s cost, quality and time to market. Procedia Computer Science 79 (2016), 8–15.
[15]
Pedro Ivo Pereira Lancellotta, Heryck Michael dos Santos Barbosa, João Gabriel Castro Santos, Klirssia Matos Isaac Sahdo, and Janislley Oliveira De Sousa. 2022. An Industry Case Study: Methodology Application to the Reviewing Process on Android Releases Homologation. In Anais Estendidos do XIII Congresso Brasileiro de Software: Teoria e Prática. SBC, 13–16.
[16]
Mario Linares-Vásquez, Carlos Bernal-Cárdenas, Kevin Moran, and Denys Poshyvanyk. 2017. How do developers test android applications?. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 613–622.
[17]
Mika V Mäntylä, Bram Adams, Foutse Khomh, Emelie Engström, and Kai Petersen. 2015. On rapid releases and software testing: a case study and a semi-systematic literature review. Empirical Software Engineering 20 (2015), 1384–1425.
[18]
Ke Mao, Mark Harman, and Yue Jia. 2017. Robotic Testing of Mobile Apps for Truly Black-Box Automation. IEEE Software 34, 2 (2017), 11–16. https://doi.org/10.1109/MS.2017.49
[19]
Leckraj Nagowah and Gayeree Sowamber. 2012. A novel approach of automation testing on mobile devices. In 2012 international conference on computer & information science (ICCIS), Vol. 2. IEEE.
[20]
Dudekula Mohammad Rafi, Katam Reddy Kiran Moses, Kai Petersen, and Mika V. Mäntylä. 2012. Benefits and limitations of automated software testing: Systematic literature review and practitioner survey. (2012), 36–42. https://doi.org/10.1109/IWAST.2012.6228988
[21]
RM Sharma. 2014. Quantitative analysis of automation and manual testing. International journal of engineering and innovative technology 4, 1 (2014).
[22]
Ian Sommerville. 2016. Software Engineering (10 ed.). Addison-Wesley, Harlow, England.
[23]
Yuqing Wang, Mika V Mäntylä, Zihao Liu, and Jouni Markkula. 2022. Test automation maturity improves product quality—Quantitative study of open source projects using continuous integration. Journal of Systems and Software 188 (2022), 111259.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
SAST '23: Proceedings of the 8th Brazilian Symposium on Systematic and Automated Software Testing
September 2023
133 pages
ISBN:9798400716294
DOI:10.1145/3624032
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Android Releases
  2. Android Tests
  3. Google Tests
  4. Quality Assurance
  5. Robotic Arm
  6. Software Testing
  7. Test Automation

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

SAST 2023

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 25
    Total Downloads
  • Downloads (Last 12 months)12
  • Downloads (Last 6 weeks)3
Reflects downloads up to 03 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media