More Web Proxy on the site http://driver.im/

research-article

Testing cross-platform mobile app development frameworks

Authors:

Nader Boushehrinejadmoradi,

Vinod Ganapathy,

Santosh Nagarakatte,

Liviu IftodeAuthors Info & Claims

ASE '15: Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering

Pages 441 - 451

https://doi.org/10.1109/ASE.2015.21

Published: 09 November 2015 Publication History

Abstract

Mobile app developers often wish to make their apps available on a wide variety of platforms, e.g., Android, iOS, and Windows devices. Each of these platforms uses a different programming environment, each with its own language and APIs for app development. Small app development teams lack the resources and the expertise to build and maintain separate code bases of the app customized for each platform. As a result, we are beginning to see a number of cross-platform mobile app development frameworks. These frameworks allow the app developers to specify the business logic of the app once, using the language and APIs of a home platform (e.g., Windows Phone), and automatically produce versions of the app for multiple target platforms (e.g., iOS and Android).

In this paper, we focus on the problem of testing cross-platform app development frameworks. Such frameworks are challenging to develop because they must correctly translate the home platform API to the (possibly disparate) target platform API while providing the same behavior. We develop a differential testing methodology to identify inconsistencies in the way that these frameworks handle the APIs of the home and target platforms. We have built a prototype testing tool, called X-Checker, and have applied it to test Xamarin, a popular framework that allows Windows Phone apps to be cross-compiled into native Android (and iOS) apps. To date, X-Checker has found 47 bugs in Xamarin, corresponding to inconsistencies in the way that Xamarin translates between the semantics of the Windows Phone and the Android APIs. We have reported these bugs to the Xamarin developers, who have already committed patches for twelve of them.

References

[1]

Adobe PhoneGap. http://phonegap.com.

[2]

Apportable - Objective-C for Android. www.apportable.com.

[3]

IBM MobileFirst platform foundation. http://www-03.ibm.com/software/products/en/mobilefirstfoundation.

[4]

Json.NET - popular high-performance JSON framework for .NET. http://james.newtonking.com/json.

[5]

Mono - Cross-platform, open source .NET framework. http://www.mono-project.com.

[6]

MyAppConverter - develop once, run anywhere. http://www.myappconverter.com.

[7]

Sencha: HTML5 app development. http://www.sencha.com.

[8]

Xamarin - Mobile App Development and App Creation Software. http://xamarin.com.

[9]

Xamarin Bug 25895. Wrong exception is thrown when calling System.Globalization.CharUnicodeInfo.GetNumericValue with invalid index. https://bugzilla.xamarin.com/show_bug.cgi?id=25895.

[10]

Xamarin Bug 27901. XmlConvert.ToString returns wrong value. https://bugzilla.xamarin.com/show_bug.cgi?id=27901.

[11]

Xamarin Bug 27910. XmlParserContext constructor not throwing XmlException when it should. https://bugzilla.xamarin.com/show_bug.cgi?id=27910.

[12]

Xamarin Bug 27922. XmlConvert.ToUnit throwing wrong/inconsistent exception. https://bugzilla.xamarin.com/show_bug.cgi?id=27922.

[13]

Xamarin Bug 27982. Inconsistent behavior in DynamicAttribute.Equals. https://bugzilla.xamarin.com/show_bug.cgi?id=27982.

[14]

Xamarin Bug 28017. NameTable.Add throwing wrong/inconsistent exception. https://bugzilla.xamarin.com/show_bug.cgi?id=28017.

[15]

Xamarin Bug 28123. Inconsistent behavior in System.Xml.XmlReaderSettings.MaxCharactersInDocument. https://bugzilla.xamarin.com/show_bug.cgi?id=28123.

[16]

Xamarin Bug 28134. System.Text.EncoderFallbackException inconsistent initial state compared to .NET (two inconsistencies assigned to this bug identifier). https://bugzilla.xamarin.com/show_bug.cgi?id=28134.

[17]

Xamarin Bug 28562. Incorrect System.Numerics.Complex.Pow result. https://bugzilla.xamarin.com/show_bug.cgi?id=28562.

[18]

Xamarin Bug 28571. Incorrect behavior in System.Numerics.BigInteger. https://bugzilla.xamarin.com/show_bug.cgi?id=28571.

[19]

Xamarin Bug 28572. Incorrect/inconsistent behavior in System.Numerics.Complex.Divide. https://bugzilla.xamarin.com/show_bug.cgi?id=28572.

[20]

S. Anand, M. Naik, H. Yang, and M. Harrold. Automated concolic testing of smartphone apps. In ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE), 2012.

[21]

T. Azim and I. Neamtiu. Targeted and depth-first exploration for systematic testing of Android apps. In ACM Symposium on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA), 2013.

Digital Library

[22]

P. Brooks and A. Memon. Automated GUI testing guided by usage profiles. In IEEE/ACM International Conference on Automated Software Engineering (ASE), 2007.

Digital Library

[23]

C. Brubaker, S. Jana, B. Ray, S. Khurshid, and V. Shmatikov. Using Frankencerts for automated adversarial testing of certificate validation in SSL/TLS implementations. In IEEE Symposium on Security and Privacy (Oakland), 2014.

[24]

D. Brumley, J. Caballero, Z. Liang, J. Newsome, and D. Song. Towards automatic discovery of deviations in binary implementations with applications to error detection and fingerprint generation. In USENIX Security Symposium, 2007.

[25]

S. Choudhary, M. Prasad, and A. Orso. X-PERT: Accurate identification of cross-browser issues in Web applications. In International Conference on Software Engineering (ICSE), 2013.

[26]

S. R. Choudhary, M. R. Prasad, and A. Orso. Cross-platform feature matching for Web applications. In International Symposium on Software Testing and Analysis (ISSTA), 2014.

[27]

C. Csallner and Y. Smaragdakis. JCrasher: An automatic robustness tester for Java. Software-Practice and Experience, 34(11), 2004.

[28]

J. E. Forrester and B. P. Miller. An empirical study of the robustness of Windows NT applications using random testing. In USENIX Windows Systems Symposium, 2000.

[29]

A. Gokhale, V. Ganapathy, and Y. Padmanaban. Inferring likely mappings between APIs. In International Conference on Software Engineering (ICSE), 2013.

[30]

S. Guthrie, T. Myerson, and S. Nadella. Day one keynote presentation, Microsoft Build developer conference, April 2015. http://channel9.msdn.com/Events/Build/2015/KEY01.

[31]

H. Heikotter, T. Majchrzak, and H. Kuchen. Cross-platform model-driven development of mobile applications with MD<sup>2</sup>. In ACM Symposium on Applied Computing (SAC), 2013.

[32]

A. Machiry, R. Tahiliani, and M. Naik. Dynodroid: An input generation system for Android apps. In ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE), 2013.

[33]

W. M. McKeeman. Differential testing for software. Digital Technical Journal, 10(1), December 1998.

[34]

B. P. Miller, L. Fredriksen, and B. So. An empirical study of the reliability of UNIX utilities. Communications of the ACM (CACM), 33(12), December 1990.

Digital Library

[35]

B. Nguyen, B. Robbins, I. Banerjee, and A. Memon. GUITAR: An innovative tool for automated testing of GUI-driven software. Journal of Automated Software Engineering, 21(1), 2014.

[36]

C. Pacheco, S. K. Lahiri, M. Ernst, and T. Ball. Feedback-directed random test generation. In International Conference on Software Engineering (ICSE), 2007.

[37]

M. Pradel, P. Schuh, G. Necula, and K. Sen. Event-break: Analyzing the responsiveness of user interfaces through performance-guided test generation. In ACM Symposium on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA), 2014.

Digital Library

[38]

M. P. Robillard, E. Bodden, D. Kawrykow, M. Mezini, and T. Ratchford. Automated API property inference techniques. IEEE Transactions on Software Engineering (TSE), 39(5), May 2013.

[39]

V. Srivastava, M. D. Bond, K. S. McKinley, and V. Shmatikov. A security policy oracle: Detecting security holes using multiple API implementation. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2011.

[40]

C. Velazco. Microsoft invites Android and iOS apps to join Windows 10, April 2015. http://www.engadget.com/2015/04/29/android-ios-apps-on-windows-10.

[41]

X. Yang, Y. Chen, E. Eide, and J. Regehr. Finding and understanding bugs in C compilers. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2011.

[42]

H. Zhong, S. Thummalapenta, and T. Xie. Exposing behavioral differences in cross-language API mapping relations. In International Conference on Fundamental Approaches to Software Engineering (FASE), 2013.

[43]

H. Zhong, S. Thummalapenta, T. Xie, L. Zhang, and Q. Wang. Mining API mapping for language migration. In International Conference on Software Engineering (ICSE), 2010.

Cited By

Martinez MTilevich E(2019)Two datasets of questions and answers for studying the development of cross-platform mobile applications using Xamarin frameworkProceedings of the 6th International Conference on Mobile Software Engineering and Systems10.5555/3340730.3340763(162-173)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.5555/3340730.3340763
Biørn-Hansen AGrønli TGhinea G(2018)A Survey and Taxonomy of Core Concepts and Research Challenges in Cross-Platform Mobile DevelopmentACM Computing Surveys10.1145/324173951:5(1-34)Online publication date: 19-Nov-2018
https://dl.acm.org/doi/10.1145/3241739
Ma YHu ZLiu YXie TLiu XChampin PGandon FMédini LLalmas MIpeirotis P(2018)AladdinProceedings of the 2018 World Wide Web Conference10.1145/3178876.3186059(1469-1478)Online publication date: 10-Apr-2018
https://dl.acm.org/doi/10.1145/3178876.3186059

Testing cross-platform mobile app development frameworks
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis

Recommendations

Developing Accessible Mobile Applications with Cross-Platform Development Frameworks
ASSETS '21: Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility

We illustrate our experience, gained over years of involvement in multiple research and commercial projects, in developing accessible mobile apps with cross-platform development frameworks (CPDF). These frameworks allow the developers to write the app ...
Evaluating the performance of Android based Cross-Platform App Development Frameworks
ICCIP '20: Proceedings of the 6th International Conference on Communication and Information Processing

Mobile applications have become a necessity in people’s daily lives, and more are being developed at this time. To increase the efficiency in developing mobile apps, cross-platform mobile app development framework is needed. Currently there are various ...
An Evaluation Framework for Cross-Platform Mobile App Development Tools: A case analysis of Adobe PhoneGap framework
CompSysTech '16: Proceedings of the 17th International Conference on Computer Systems and Technologies 2016

The 'App economy' is a highly lucrative and competitive market for independent software vendors as it potentially offers an easy highway to reach millions of users. However, the mobile application landscape is scattered and an application developer has ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ASE '15: Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering

November 2015

935 pages

ISBN:9781509000241

General Chair:
Myra Cohen
University of Nebraska-Lincoln
,
Program Chairs:
Lars Grunske
University of Stuttgart, Germany
,
Michael Whalen
University of Minnesota

Sponsors

In-Cooperation

IEEE CS

Publisher

IEEE Press

Publication History

Published: 09 November 2015

Check for updates

Qualifiers

Research-article

Conference

ASE '15

Sponsor:

ASE '15: ACM/IEEE International Conference on Automated Software Engineering

November 9 - 15, 2015

Nebraska, Lincoln

Acceptance Rates

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
210
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)1

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Martinez MTilevich E(2019)Two datasets of questions and answers for studying the development of cross-platform mobile applications using Xamarin frameworkProceedings of the 6th International Conference on Mobile Software Engineering and Systems10.5555/3340730.3340763(162-173)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.5555/3340730.3340763
Biørn-Hansen AGrønli TGhinea G(2018)A Survey and Taxonomy of Core Concepts and Research Challenges in Cross-Platform Mobile DevelopmentACM Computing Surveys10.1145/324173951:5(1-34)Online publication date: 19-Nov-2018
https://dl.acm.org/doi/10.1145/3241739
Ma YHu ZLiu YXie TLiu XChampin PGandon FMédini LLalmas MIpeirotis P(2018)AladdinProceedings of the 2018 World Wide Web Conference10.1145/3178876.3186059(1469-1478)Online publication date: 10-Apr-2018
https://dl.acm.org/doi/10.1145/3178876.3186059

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents