More Web Proxy on the site http://driver.im/

research-article

Analyzing the analyzers: FlowDroid/IccTA, AmanDroid, and DroidSafe

Authors:

Julia RubinAuthors Info & Claims

ISSTA 2018: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis

Pages 176 - 186

https://doi.org/10.1145/3213846.3213873

Published: 12 July 2018 Publication History

Abstract

Numerous static analysis techniques have recently been proposed for identifying information flows in mobile applications. These techniques are compared to each other, usually on a set of syntactic benchmarks. Yet, configurations used for such comparisons are rarely described. Our experience shows that tools are often compared under different setup, rendering the comparisons irreproducible and largely inaccurate. In this paper, we provide a large, controlled, and independent comparison of the three most prominent static analysis tools: FlowDroid combined with IccTA, Amandroid, and DroidSafe. We evaluate all tools using common configuration setup and the same set of benchmark applications. We compare the results of our analysis to the results reported in previous studies, identify main reasons for inaccuracy in existing tools, and provide suggestions for future research.

References

[1]

2017. DroidBench Benchmark Suite. https://github.com/ secure-software-engineering/DroidBench. 2017. ICC-Bench Benchmark Suite. https://github.com/fgwei/ICC-Bench. 2017. PLDI’14 Artifact Evaluation. https://github.com/ secure-software-engineering/soot-infoflow-android/wiki/PLDI’ 14-Artifact-Evaluation. 2017. Fortify SCA. https://software.microfocus.com/en-us/solutions/ enterprise-security. 2017. IBM AppScan Source. https://www.ibm.com/us-en/marketplace/ ibm-appscan-source.

[2]

Steven Arzt. 2017. Static Data Flow Analysis for Android Applications. Ph.D. Dissertation. Darmstadt University of Technology, Germany.

[3]

Steven Arzt, Siegfried Rasthofer, Christian Fritz, Eric Bodden, Alexandre Bartel, Jacques Klein, Yves Le Traon, Damien Octeau, and Patrick McDaniel. 2014. Flow-Droid: Precise Context, Flow, Field, Object-sensitive and Lifecycle-aware Taint Analysis for Android Apps. In Proc. of PLDI’14. 259–269.

Digital Library

[4]

Paulo Barros, René Just, Suzanne Millstein, Paul Vines, Werner Dietl, Marcelo d’Amorim, and Michael D. Ernst. 2015. Static Analysis of Implicit Control Flow: Resolving Java Reflection and Android Intents (T). In Proc. of ASE’15. 669–679.

[5]

Alexandre Bartel, Jacques Klein, Yves Le Traon, and Martin Monperrus. 2012. Dexpler: Converting Android Dalvik Bytecode to Jimple for Static Analysis with Soot. In Proceedings of the ACM SIGPLAN International Workshop on State of the Art in Java Program Analysis. 27–38.

Digital Library

[6]

Shweta Bhandari, Wafa Ben Jaballah, Vineeta Jain, Vijay Laxmi, Akka Zemmari, Manoj Singh Gaur, Mohamed Mosbah, and Mauro Conti. 2017. Android Inter-app Communication Threats and Detection Techniques. Computers & Security 70 (2017), 392–421.

[7]

Antonio Bianchi, Jacopo Corbetta, Luca Invernizzi, Yanick Fratantonio, Christopher Kruegel, and Giovanni Vigna. 2015. What the App is That? Deception and Countermeasures in the Android User Interface. In Proc. of S&P 2015. 931–948.

Digital Library

[8]

Yanick Fratantonio, Aravind Machiry, Antonio Bianchi, Christopher Kruegel, and Giovanni Vigna. 2015. CLAPP: Characterizing Loops in Android Applications. In Proc. of ESEC/FSE 2015. 687–697.

Digital Library

[9]

Michael I Gordon, Deokhwan Kim, Jeff H Perkins, Limei Gilham, Nguyen Nguyen, and Martin C Rinard. 2015. Information Flow Analysis of Android Applications in DroidSafe. In Proc. of NDSS’15.

[10]

Johannes Hoffmann, Teemu Rytilahti, Davide Maiorca, Marcel Winandy, Giorgio Giacinto, and Thorsten Holz. 2016. Evaluating Analysis Tools for Android Apps: Status Quo and Robustness Against Obfuscation. In Proc. of CODASPY’16. 139– 141.

Digital Library

[11]

John Jorgensen. 2003. Improving the Precision and Correctness of Exception Analysis in Soot. Technical Report 2003-3. McGill University, Canada.

[12]

George Kastrinis and Yannis Smaragdakis. 2013. Efficient and Effective Handling Of Exceptions in Java Points-to Analysis. In Proc. of CC’13. 41–60.

Digital Library

[13]

William Klieber, Lori Flynn, Amar Bhosale, Limin Jia, and Lujo Bauer. 2014. Android Taint Flow Analysis for App Sets. In Proc. of PLDI Workshop on Software Analysis (SOAP’15). 1–6.

Digital Library

[14]

Ondřej Lhoták and Laurie Hendren. 2003. Scaling Java Points-to Analysis Using SPARK. In Proc. of CC’03. 153–169.

Digital Library

[15]

Li Li, Alexandre Bartel, Tegawendé F Bissyandé, Jacques Klein, Yves Le Traon, Steven Arzt, Siegfried Rasthofer, Eric Bodden, Damien Octeau, and Patrick Mc-Daniel. 2015. IccTA: Detecting Inter-component Privacy Leaks in Android Apps. In Proc. of ICSE’15.

Digital Library

[16]

Li Li, Alexandre Bartel, Tegawendé F. Bissyandé, Jacques Klein, and Yves Le Traon. 2015. ApkCombiner: Combining Multiple Android Apps to Support Inter-App Analysis. In Proc. of SEC’15. 513–527.

[17]

Li Li, Tegawendé F Bissyandé, Mike Papadakis, Siegfried Rasthofer, Alexandre Bartel, Damien Octeau, Jacques Klein, and Le Traon. 2017. Static Analysis of Android Apps: A Systematic Literature Review. Information & Software Technology 88 (2017), 67–95.

Digital Library

[18]

Li Li, Jun Gao, Médéric Hurier, Pingfan Kong, Tegawendé F Bissyandé, Alexandre Bartel, Jacques Klein, and Yves Le Traon. 2017. AndroZoo++: Collecting Millions of Android Apps and Their Metadata for the Research Community. The Computing Research Repository abs/1709.05281 (2017).

[19]

Julia Rubin Lina Qiu, Yingying Wang. 2018. Supplementary Materials. https: //resess.github.io/PaperAppendices/ISSTA2018.html.

[20]

Zhang Mu, Duan Yue, Yin Heng, and Zhao Zhiruo. 2014. Semantics-Aware Android Malware Classification Using Weighted Contextual API Dependency Graphs. In Proc. of CCS’14. 1105–1116.

Digital Library

[21]

Damien Octeau, Somesh Jha, Matthew Dering, Patrick McDaniel, Alexandre Bartel, Li Li, Jacques Klein, and Yves Le Traon. 2016. Combining Static Analysis with Probabilistic Models to Enable Market-scale Android Inter-component Analysis. In Proc. POPL’16. 469–484.

Digital Library

[22]

Damien Octeau, Daniel Luchaup, Matthew Dering, Somesh Jha, and Patrick McDaniel. 2015. Composite Constant Propagation: Application to Android Inter-Component Communication Analysis. In Proc. of ICSE’15. 77–88.

Digital Library

[23]

Damien Octeau, Patrick McDaniel, Somesh Jha, Alexandre Bartel, Eric Bodden, Jacques Klein, and Yves Le Traon. 2013. Effective Inter-Component Communication Mapping in Android: An Essential Step Towards Holistic Security Analysis. In Proc. of USENIX Security 2013. 543–558.

Digital Library

[24]

David J. Pearce, Paul H.J. Kelly, and Chris Hankin. 2007. Efficient Field-sensitive Pointer Analysis of C. ACM Transactions on Programming Languages and Systems 30, 1 (2007).

Digital Library

[25]

Siegfried Rasthofer, Steven Arzt, and Eric Bodden. 2014. A Machine-learning Approach for Classifying and Categorizing Android Sources and Sinks. In Proc. of NDSS’14.

[26]

Bradley Reaves, Jasmine Bowers, Sigmund Albert Gorski III, Olabode Anise, Rahul Bobhate, Raymond Cho, Hiranava Das, Sharique Hussain, Hamza Karachiwala, Nolen Scaife, et al. 2016. *droid: Assessment and Evaluation of Android Application Analysis Tools. Comput. Surveys 49, 3 (2016), 55.

Digital Library

[27]

Alejandro Russo, Andrei Sabelfeld, and Keqin Li. 2010. Implicit Flows in Malicious and Nonmalicious Code. Logics and Languages for Reliability and Security 25 (2010), 301–322.

[28]

Alireza Sadeghi, Hamid Bagheri, Joshua Garcia, and Sam Malek. 2017. A Taxonomy and Qualitative Comparison of Program Analysis Techniques for Security Assessment of Android Software. IEEE Transactions on Software Engineering 43, 6 (2017), 492–530.

Digital Library

[29]

Suzanna Schmeelk and Junfeng Yang andAlfred V. Aho. 2015. Android Malware Static Analysis Techniques. In Proc. of CISR’15. 5:1–5:8.

Digital Library

[30]

Yannis Smaragdakis, George Balatsouras, et al. 2015. Pointer Analysis. Foundations and Trends® in Programming Languages 2, 1 (2015), 1–69.

Digital Library

[31]

Sufatrio, Darell J. J. Tan, Tong-Wei Chua, and Vrizlynn L. L. Thing. 2015. Securing Android: A Survey, Taxonomy, and Challenges. Comput. Surveys 47, 4 (2015), 58:1–58:45.

Digital Library

[32]

Raja Vallée-Rai, Phong Co, Etienne Gagnon, Laurie Hendren, Patrick Lam, and Vijay Sundaresan. 1999. Soot - a Java Bytecode Optimization Framework. In Proc. of CASCON’99.

Digital Library

[33]

Fengguo Wei, Sankardas Roy, Xinming Ou, et al. 2014. Amandroid: A Precise and General Inter-component Data Flow Analysis Framework for Security Vetting of Android Apps. In Proc. of CCS’14. 1329–1341.

Digital Library

[34]

Fengguo Wei, Sankardas Roy, Xinming Ou, and Robby. 2017. Amandroid: A Precise and General Inter-component Data Flow Analysis Framework for Security Vetting of Android Apps. Technical Report 2017-4. University of South Florida, USA.

[35]

John Whaley, Dzintars Avots, Michael Carbin, and Monica S Lam. 2005. Using Datalog with Binary Decision Diagrams for Program Analysis. In Proc. of APLAS’05. 97–118.

Digital Library

Cited By

Fan MShi JWang YYu LZhang XWang HJin WLiu TFilkov VRay BZhou M(2024)Giving without Notifying: Assessing Compliance of Data Transmission in Android AppsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695528(1595-1606)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695528
Cao JGuo FQu Y(2024)JNFuzz-Droid: A Lightweight Fuzzing and Taint Analysis Framework for Android Native Code2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00033(255-266)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00033
Nirumand AZamani BLadani B(2024)A comprehensive framework for inter-app ICC security analysis of Android appsAutomated Software Engineering10.1007/s10515-024-00439-831:2Online publication date: 4-Jun-2024
https://doi.org/10.1007/s10515-024-00439-8
Show More Cited By

Index Terms

Analyzing the analyzers: FlowDroid/IccTA, AmanDroid, and DroidSafe
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Empirical software validation

Recommendations

Amandroid: A Precise and General Inter-component Data Flow Analysis Framework for Security Vetting of Android Apps

We present a new approach to static analysis for security vetting of Android apps and a general framework called Amandroid. Amandroid determines points-to information for all objects in an Android app component in a flow and context-sensitive (user-...
FlowDroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for Android apps
PLDI '14: Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation

Today's smartphones are a ubiquitous source of private and confidential data. At the same time, smartphone users are plagued by carelessly programmed apps that leak important data by accident, and by malicious apps that exploit their given privileges to ...
FlowDroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for Android apps
PLDI '14

Today's smartphones are a ubiquitous source of private and confidential data. At the same time, smartphone users are plagued by carelessly programmed apps that leak important data by accident, and by malicious apps that exploit their given privileges to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2018: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis

July 2018

379 pages

ISBN:9781450356992

DOI:10.1145/3213846

General Chair:
Frank Tip
Northeastern University, USA
,
Program Chair:
Eric Bodden
University of Paderborn, Germany / Fraunhofer IEM, Germany

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Distinguished Paper

Author Tags

Qualifiers

Research-article

Conference

ISSTA '18

Sponsor:

SIGSOFT

ISSTA '18: International Symposium on Software Testing and Analysis

July 16 - 21, 2018

Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

61
Total Citations
View Citations
968
Total Downloads

Downloads (Last 12 months)99
Downloads (Last 6 weeks)9

Reflects downloads up to 26 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fan MShi JWang YYu LZhang XWang HJin WLiu TFilkov VRay BZhou M(2024)Giving without Notifying: Assessing Compliance of Data Transmission in Android AppsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695528(1595-1606)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695528
Cao JGuo FQu Y(2024)JNFuzz-Droid: A Lightweight Fuzzing and Taint Analysis Framework for Android Native Code2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00033(255-266)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00033
Nirumand AZamani BLadani B(2024)A comprehensive framework for inter-app ICC security analysis of Android appsAutomated Software Engineering10.1007/s10515-024-00439-831:2Online publication date: 4-Jun-2024
https://doi.org/10.1007/s10515-024-00439-8
Vineetha BSuryaprasad JShylaja SHonnavalli P(2024)A Deep Dive into Deep Learning-Based Adversarial Attacks and Defenses in Computer Vision: From a Perspective of CybersecurityIntelligent Sustainable Systems10.1007/978-981-99-7569-3_28(341-356)Online publication date: 16-Feb-2024
https://doi.org/10.1007/978-981-99-7569-3_28
Nan YWang XXing LLiao XWu RWu JZhang YWang XCalandrino JTroncoso C(2023)Are you spying on me?Proceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620610(6665-6682)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620610
Liu PZhao YFazzini MCai HGrundy JLi L(2023)Automatically Detecting Incompatible Android APIsACM Transactions on Software Engineering and Methodology10.1145/362473733:1(1-33)Online publication date: 23-Nov-2023
https://dl.acm.org/doi/10.1145/3624737
Ahmed KWang YLis MRubin JChandra SBlincoe KTonella P(2023)ViaLin: Path-Aware Dynamic Taint Analysis for AndroidProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616330(1598-1610)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616330
Li KChen SFan LFeng RLiu HLiu CLiu YChen YChandra SBlincoe KTonella P(2023)Comparison and Evaluation on Static Application Security Testing (SAST) Tools for JavaProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616262(921-933)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616262
Mordahl AJust RFraser G(2023)Automatic Testing and Benchmarking for Configurable Static Analysis ToolsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3605232(1532-1536)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3605232
Neupane PThakur M(2023)Variational Study of the Impact of Call Graphs on Precision of Android Taint AnalysisProceedings of the 16th Innovations in Software Engineering Conference10.1145/3578527.3578545(1-5)Online publication date: 23-Feb-2023
https://dl.acm.org/doi/10.1145/3578527.3578545
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents