More Web Proxy on the site http://driver.im/

research-article

Identifying and documenting false positive patterns generated by static code analysis tools

Authors:

Zachary P. Reynolds,

Abhinandan B. Jayanth,

Adam A. Porter,

Rajeev R. Raje,

James H. HillAuthors Info & Claims

SER&IP '17: Proceedings of the 4th International Workshop on Software Engineering Research and Industrial Practice

Pages 55 - 61

https://doi.org/10.1109/SER-IP.2017.20

Published: 20 May 2017 Publication History

Abstract

This paper presents our results from identifying and documenting false positives generated by static code analysis tools. By false positives, we mean a static code analysis tool generates a warning message, but the warning message is not really an error. The goal of our study is to understand the different kinds of false positives generated so we can (1) automatically determine if an error message is truly indeed a true positive, and (2) reduce the number of false positives developers and testers must triage. We have used two open-source tools and one commercial tool in our study. The results of our study have led to 14 core false positive patterns, some of which we have confirmed with static code analysis tool developers.

References

[1]

P. Louridas, "Static code analysis," IEEE Software, vol. 23, no. 4, pp. 58--61, 2006.

Digital Library

[2]

"The MITRE corporation," https://www.mitre.org/.

[3]

"Common Weakness Enumeration frequently asked questions," http://cwe.mitre.org/about/faq.html, accessed: 2016-09-13.

[4]

Wikipedia, "List of tools for static code analysis --- Wikipedia, the free encyclopedia," 2016, {Online; accessed 13-September-2016}. {Online}. Available: https://en.wikipedia.org/w/index.php?title=List_of_tools_for_static_code_analysis&oldid=739038439

[5]

"Microsoft code analysis tool .NET (CAT.NET) v1 CTP - 32 bit," https://www.microsoft.com/en-us/download/details.aspx?id=19968, accessed: 2016-09-18.

[6]

"Findbugs - find bugs in Java programs," http://findbugs.sourceforge.net/, accessed: 2016-09-19.

[7]

"A look at CWE coverage across open source and commercial static analysis tools," https://codedx.com/a-look-at-cwe-coverage-across-open-source-and-commercial-static-analysis-tools/?v=7516fd43adaa, accessed: 2016-09-13.

[8]

M. Nadeem, B. J. Williams, and E. B. Allen, "High false positive detection of security vulnerabilities: A case study," in Proceedings of the 50th Annual Southeast Regional Conference. ACM, 2012, pp. 359--360.

Digital Library

[9]

N. Antunes and M. Vieira, "Comparing the effectiveness of penetration testing and static code analysis on the detection of SQL injection vulnerabilities in web services," in Dependable Computing, 2009. PRDC'09. 15th IEEE Pacific Rim International Symposium on. IEEE, 2009, pp. 301--306.

Digital Library

[10]

M. Zitser, R. Lippmann, and T. Leek, "Testing static analysis tools using exploitable buffer overflows from open source code," in ACM SIGSOFT Software Engineering Notes, vol. 29, no. 6. ACM, 2004, pp. 97--106.

Digital Library

[11]

N. Ayewah, W. Pugh, J. D. Morgenthaler, J. Penix, and Y. Zhou, "Evaluating static analysis defect warnings on production software," in Proceedings of the 7th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, ser. PASTE '07. New York, NY, USA: ACM, 2007, pp. 1--8. {Online}. Available

Digital Library

[12]

N. Ayewah, D. Hovemeyer, J. D. Morgenthaler, J. Penix, and W. Pugh, "Using static analysis to find bugs," IEEE software, vol. 25, no. 5, pp. 22--29, 2008.

Digital Library

[13]

"Software assurance reference dataset," https://samate.nist.gov/SRD/testsuite.php, accessed: 2016-09-19.

[14]

The Common Weakness Enumeration (CWE) Initiative, MITRE Corporation. http://cwe.mitre.org/.

[15]

A. Delaitre, B. Stivalet, E. Fong, and V. Okun, "Evaluating bug finders: Test and measurement of static code analyzers," in Proceedings of the First International Workshop on Complex faUlts and Failures in LargE Software Systems. IEEE Press, 2015, pp. 14--20.

Digital Library

[16]

L. M. R. Velicheti, D. C. Feiock, M. Peiris, R. Raje, and J. H. Hill, "Towards modeling the behavior of static code analysis tools," in Proceedings of the 9th Annual Cyber and Information Security Research Conference, ser. CISR '14. New York, NY, USA: ACM, 2014, pp. 17--20. {Online}. Available

Digital Library

[17]

"Valgrind," http://valgrind.org/, accessed: 2016-09-19.

[18]

M. Marenchino, "Source code reduction to summarize false positives," Ph.D. dissertation, 2015.

[19]

J. O. Coplien, "Software design patterns," in Encyclopedia of Computer Science. Chichester, UK: John Wiley and Sons Ltd., pp. 1604--1606. {Online}. Available: http://dl.acm.org/citation.cfm?id=1074100.1074801

[20]

"Java development kit 8," http://www.oracle.com/technetwork/java/javase/downloads/index.html, accessed: 2016-09-19.

[21]

"Glassfish open source edition 4.1.1," http://www.oracle.com/technetwork/java/javaee/downloads/index.html, accessed: 2016-09-19.

[22]

U. Yuksel and H. Sozer, "Automated classification of static code analysis alerts: A case study," in Software Maintenance (ICSM), 2013 29th IEEE International Conference on, Sept 2013, pp. 532--535.

Digital Library

[23]

"Vestel electronics," http://vestelinternational.com/en/, accessed: 2016-09-23.

[24]

V. R. L. de Mendonca, C. L. Rodrigues, F. A. A. de MN Soares, and A. M. R. Vincenzi, "Static analysis techniques and tools: A systematic mapping study," 2013.

[25]

V. R. Basili and R. W. Selby, "Comparing the effectiveness of software testing strategies," IEEE transactions on software engineering, no. 12, pp. 1278--1296, 1987.

Digital Library

[26]

"Coreutils - GNU core utilities," https://www.gnu.org/software/coreutils/coreutils.html, accessed: 2017-02-25.

[27]

"POCO C++ libraries," https://pocoproject.org/, accessed: 2017-02-25.

Cited By

Guo ZTan TLiu SLiu XLai WYang YLi YChen LDong WZhou Y(2023)Mitigating False Positive Static Analysis Warnings: Progress, Challenges, and OpportunitiesIEEE Transactions on Software Engineering10.1109/TSE.2023.332966749:12(5154-5188)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TSE.2023.3329667
Choi YNam J(2023)WINEInformation and Software Technology10.1016/j.infsof.2022.107109155:COnline publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1016/j.infsof.2022.107109
Tahaei MVaniea KBeznosov KWolters MKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Security Notifications in Static Analysis Tools: Developers’ Attitudes, Comprehension, and Ability to Act on ThemProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445616(1-17)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445616
Show More Cited By

Identifying and documenting false positive patterns generated by static code analysis tools
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis

Recommendations

Learning a classifier for false positive error reports emitted by static code analysis tools
MAPL 2017: Proceedings of the 1st ACM SIGPLAN International Workshop on Machine Learning and Programming Languages

The large scale and high complexity of modern software systems make perfectly precise static code analysis (SCA) infeasible. Therefore SCA tools often over-approximate, so not to miss any real problems. This, however, comes at the expense of raising ...
The False Dichotomy between Positive and Negative Affect in Game Play
CHI PLAY '15: Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play

Most of the time games make us happy, but sometimes they are frustrating or make us feel sad. They allow us to experience pleasure, success and joy, but they can also yield feelings of frustration, failure, or sorrow from darker themes. In games, we can ...
Difficult XSS Code Patterns for Static Code Analysis Tools
Computer Security
Abstract
We present source code patterns that are difficult for modern static code analysis tools. Our study comprises 50 different open source projects in both a vulnerable and a fixed version for XSS vulnerabilities reported with CVE IDs over a period of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SER&IP '17: Proceedings of the 4th International Workshop on Software Engineering Research and Industrial Practice

May 2017

79 pages

ISBN:9781538627976

Program Chairs:
Rakesh Shukla
Infosys Limited, Bangalore, India
,
Sagar Sen
Simula Research Laboratory, Norway
,
Judith Bishop
Stellenbosch University, South Africa
,
Karin Breitman
Pontifical Catholic University of Rio de Janeiro, Brazil

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society
SADIO: SADIO

Publisher

IEEE Press

Publication History

Published: 20 May 2017

Check for updates

Qualifiers

Research-article

Conference

SCF '17

Sponsor:

SIGSOFT
IEEE-CS
SADIO

SCF '17: ACM Symposium on Computational Fabrication

May 20 - 28, 2017

Buenos Aires, Argentina

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
133
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Guo ZTan TLiu SLiu XLai WYang YLi YChen LDong WZhou Y(2023)Mitigating False Positive Static Analysis Warnings: Progress, Challenges, and OpportunitiesIEEE Transactions on Software Engineering10.1109/TSE.2023.332966749:12(5154-5188)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TSE.2023.3329667
Choi YNam J(2023)WINEInformation and Software Technology10.1016/j.infsof.2022.107109155:COnline publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1016/j.infsof.2022.107109
Tahaei MVaniea KBeznosov KWolters MKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Security Notifications in Static Analysis Tools: Developers’ Attitudes, Comprehension, and Ability to Act on ThemProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445616(1-17)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445616
Khaled LAbdelbaki N(2020)Evaluation of Software Static AnalyzersProceedings of the 9th International Conference on Software and Information Engineering10.1145/3436829.3436835(11-17)Online publication date: 11-Nov-2020
https://dl.acm.org/doi/10.1145/3436829.3436835
Imtiaz NRahman AFarhana EWilliams LStorey MAdams BHaiduc S(2019)Challenges with responding to static analysis tool alertsProceedings of the 16th International Conference on Mining Software Repositories10.1109/MSR.2019.00049(245-249)Online publication date: 26-May-2019
https://dl.acm.org/doi/10.1109/MSR.2019.00049
Koc USaadatpanah PFoster JPorter AShpeisman TGottschlich J(2017)Learning a classifier for false positive error reports emitted by static code analysis toolsProceedings of the 1st ACM SIGPLAN International Workshop on Machine Learning and Programming Languages10.1145/3088525.3088675(35-42)Online publication date: 18-Jun-2017
https://dl.acm.org/doi/10.1145/3088525.3088675

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents