More Web Proxy on the site http://driver.im/

research-article

An historical examination of open source releases and their vulnerabilities

Authors:

Liqun ChenAuthors Info & Claims

CCS '12: Proceedings of the 2012 ACM conference on Computer and communications security

Pages 183 - 194

https://doi.org/10.1145/2382196.2382218

Published: 16 October 2012 Publication History

Abstract

This paper examines historical releases of Sendmail, Postfix, Apache httpd and OpenSSL by using static source code analysis and the entry-rate in the Common Vulnerabilities and Exposures dictionary (CVE) for a release, which we take as a measure of the rate of discovery of exploitable bugs. We show that the change in number and density of issues reported by the source code analyzer is indicative of the change in rate of discovery of exploitable bugs for new releases --- formally we demonstrate a statistically significant correlation of moderate strength. The strength of the correlation is an artifact of other factors such as the degree of scrutiny: the number of security analysts investigating the software. This also demonstrates that static source code analysis can be used to make some assessment of risk even when constraints do not permit human review of the issues identified by the analysis.

We find only a weak correlation between absolute values measured by the source code analyzer and rate of discovery of exploitable bugs, so in general it is unsafe to use absolute values of number of issues or issue densities to compare different applications or software. Our results demonstrate that software quality, as measured by the number of issues, issue density or number of exploitable bugs, does not always improve with each new release. However, generally the rate of discovery of exploitable bugs begins to drop three to five years after the initial release.

References

[1]

O. H. Alhazmi and Y. K. Malaiya. Quantitative vulnerability assessment of systems software. In Proceedings of the IEEE Reliability and Maintainability Symposium, pp615--620, 2005.

[2]

Apache release history. http://www.apachehaus.com/.

[3]

N. Ayewah, W. Pugh, J. D. Morgenthaler, J. Penix, and Y. Zhou. Evaluating static analysis defect warnings on production software. In PASTE '07 Proceedings of the 7th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering, pp1--8, 2007.

Digital Library

[4]

F. P. Brooks. The Mythical Man Month and Other Essays on Software Engineering. Addison Wesley, 1975, 1995(2nd Ed.).

Digital Library

[5]

B. Chess and J. West. Secure Programming with Static Analysis. Pearson Education Inc., Boston, Massachusetts, 2007.

Digital Library

[6]

B. V. Chess. Improving computer security using extended static checking. In Proceedings of IEEE Symposium on Security and Privacy, pp160--173, 2002.

Digital Library

[7]

S. Clark, S. Frei, M. Blaze, and J. Smith. Familiarity breeds contempt: the honeymoon effect and the role of legacy code in zero-day vulnerabilities. In ACSAC '10: Proceedings of the 26th Annual Computer Security Applications Conference, pp251--260, December 2010.

Digital Library

[8]

M. Doyle and J. Walden. An empirical study of the evolution of PHP web application security. In International Workshop On Security Measurments and Metrics, MetriSec, 2011.

Digital Library

[9]

N. Falliere, L. O. Murchu, and E. Chien. W32.stuxnet dossier, version 1.4 (February 2011). http://www.symantec.com/.

[10]

R. Gopalakrishna and E. H. Spafford. A trend analysis of vulnerabilities. In Technical Report 2005-05, CERIAS, Purdue University, May 2005.

[11]

W. Landi. Undecidability of static analysis. ACM Letters on Programming Languages and Systems (LOPLAS), 4(1):323--337, December 1992.

Digital Library

[12]

The common vulnerabilities and exposures dictionary. http://cve.mitre.org/.

[13]

N. Nagappan and T. Ball. Static analysis tools as early indicators of pre-release defect density. In ICSE '05 Proceedings of the 27th international conference on Software engineering, pp580--586, 2005.

Digital Library

[14]

National vulnerability database. http://nvd.nist.gov/.

[15]

October 2011 web server survey. http://news.netcraft.com/.

[16]

F. Nielson, H. R. Nielson, and C. Hankin. Principles of Program Analysis. Springer-Verlag, Berlin, Germany, 2005, 2nd Ed.

Digital Library

[17]

V. Okun, W. Guthrie, R.Gaucher, and P. Black. Effect of static analysis tools on software security: preliminary investigation. In QoP '07: Proceedings of the 2007 ACM workshop on Quality of protection pp1--5, October 2007.

Digital Library

[18]

A. Ozment. The likelihood of vulnerability rediscovery and the social utility of vulnerability hunting. In Workshop on the Economics of Information Security (WEIS), Cambridge, MA, USA, June 2005.

[19]

A. Ozment. Improving vulnerability discovery models. In QoP '07: Proceedings of the 2007 ACM workshop on Quality of protection, pp6--11, October 2007.

Digital Library

[20]

A. Ozment and S. E. Schechter. Milk or wine: does software security improve with age? In Proceedings of the 15th conference on USENIX Security Symposium - Volume 15, pp93--104, 2006.

Digital Library

[21]

E. Rescorla. Is finding security holes a good idea? IEEE Security & Privacy, 3(1):14--19, February 2005.

Digital Library

[22]

H. Rice. Classes of recursively enumerable sets and their decision problems. Trans. Amer. Math. Soc., 74(2):358--366, March 1953.

[23]

N. Rutar, C. B. Almazan, and J. S. Foster. A comparison of bug finding tools for java. In ISSRE '04 Proceedings of the 15th International Symposium on Software Reliability Engineering, pp245--256, October 2007.

Digital Library

[24]

Securityfocus vulnerability database. http://www.securityfocus.com/vulnerabilities.

[25]

E. H. Spafford. The internet worm program: An analysis. ACM SIGCOMM Computer Communication Review, 19(1):17--57, January 1989.

Digital Library

Cited By

Zakharov AGladkikh K(2024)Characteristics and Trends of Zero-Day Vulnerabilities in Open-Source Code2024 International Russian Automation Conference (RusAutoCon)10.1109/RusAutoCon61949.2024.10694228(498-502)Online publication date: 8-Sep-2024
https://doi.org/10.1109/RusAutoCon61949.2024.10694228
Dam TNeumaier S(2024)Towards Measuring Vulnerabilities and Exposures in Open-Source PackagesData Science—Analytics and Applications10.1007/978-3-031-42171-6_2(13-19)Online publication date: 4-Jan-2024
https://doi.org/10.1007/978-3-031-42171-6_2
Fourné MWermke DEnck WFahl SAcar Y(2023)It’s like flossing your teeth: On the Importance and Challenges of Reproducible Builds for Software Supply Chain Security2023 IEEE Symposium on Security and Privacy (SP)10.1109/SP46215.2023.10179320(1527-1544)Online publication date: May-2023
https://doi.org/10.1109/SP46215.2023.10179320
Show More Cited By

Index Terms

An historical examination of open source releases and their vulnerabilities
1. Security and privacy
2. Social and professional topics
  1. Computing / technology policy
    1. Computer crime

Recommendations

Revisiting reopened bugs in open source software systems
Abstract
Reopened bugs can degrade the overall quality of a software system since they require unnecessary rework by developers. Moreover, reopened bugs also lead to a loss of trust in the end-users regarding the quality of the software. Thus, predicting ...
Tracking patches for open source software vulnerabilities
ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Open source software (OSS) vulnerabilities threaten the security of software systems that use OSS. Vulnerability databases provide valuable information (e.g., vulnerable version and patch) to mitigate OSS vulnerabilities. There arises a growing ...
Open Source Drives Innovation

Engineers using free and open source software created many of today's most innovative products and solutions. FOSS also changed the way we develop software. IEEE Software's Open Source column first appeared in this magazine in July 2004. In this last ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '12: Proceedings of the 2012 ACM conference on Computer and communications security

October 2012

1088 pages

ISBN:9781450316514

DOI:10.1145/2382196

General Chair:
Ting Yu
North Carolina State University, USA
,
Program Chairs:
George Danezis
Microsoft Research Cambridge, UK
,
Virgil Gligor
Carnegie Mellon University, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS'12

Sponsor:

SIGSAC

CCS'12: the ACM Conference on Computer and Communications Security

October 16 - 18, 2012

North Carolina, Raleigh, USA

Acceptance Rates

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
1,039
Total Downloads

Downloads (Last 12 months)71
Downloads (Last 6 weeks)10

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zakharov AGladkikh K(2024)Characteristics and Trends of Zero-Day Vulnerabilities in Open-Source Code2024 International Russian Automation Conference (RusAutoCon)10.1109/RusAutoCon61949.2024.10694228(498-502)Online publication date: 8-Sep-2024
https://doi.org/10.1109/RusAutoCon61949.2024.10694228
Dam TNeumaier S(2024)Towards Measuring Vulnerabilities and Exposures in Open-Source PackagesData Science—Analytics and Applications10.1007/978-3-031-42171-6_2(13-19)Online publication date: 4-Jan-2024
https://doi.org/10.1007/978-3-031-42171-6_2
Fourné MWermke DEnck WFahl SAcar Y(2023)It’s like flossing your teeth: On the Importance and Challenges of Reproducible Builds for Software Supply Chain Security2023 IEEE Symposium on Security and Privacy (SP)10.1109/SP46215.2023.10179320(1527-1544)Online publication date: May-2023
https://doi.org/10.1109/SP46215.2023.10179320
Ye MRuocco ABuono DBottomley JFranke H(2023)Free the Turtles: Removing Nested Virtualization for Performance and Confidentiality in the Cloud2023 IEEE 16th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD60044.2023.00039(275-281)Online publication date: Jul-2023
https://doi.org/10.1109/CLOUD60044.2023.00039
Li XMoreschini SZhang ZPalomba FTaibi D(2023)The anatomy of a vulnerability database: A systematic mapping studyJournal of Systems and Software10.1016/j.jss.2023.111679201(111679)Online publication date: Jul-2023
https://doi.org/10.1016/j.jss.2023.111679
Wermke DWöhler NKlemmer JFourné MAcar YFahl S(2022)Committed to Trust: A Qualitative Study on Security & Trust in Open Source Software Projects2022 IEEE Symposium on Security and Privacy (SP)10.1109/SP46214.2022.9833686(1880-1896)Online publication date: May-2022
https://doi.org/10.1109/SP46214.2022.9833686
Li QSong JTan DWang HLiu J(2021)PDGraph: A Large-Scale Empirical Study on Project Dependency of Security Vulnerabilities2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN48987.2021.00031(161-173)Online publication date: Jun-2021
https://doi.org/10.1109/DSN48987.2021.00031
Alexopoulos NHabib SSchulz SMühlhäuser M(2020)The Tip of the IcebergACM Transactions on Privacy and Security10.1145/340611224:1(1-33)Online publication date: 28-Sep-2020
https://dl.acm.org/doi/10.1145/3406112
Koller RWilliams DYadgar GNoh S(2019)An ounce of prevention is worth a pound of cureProceedings of the 11th USENIX Conference on Hot Topics in Storage and File Systems10.5555/3357062.3357078(12-12)Online publication date: 8-Jul-2019
https://dl.acm.org/doi/10.5555/3357062.3357078
Mitropoulos CDumas MPfahl DApel SRusso A(2019)Employing different program analysis methods to study bug evolutionProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3338906.3342489(1202-1204)Online publication date: 12-Aug-2019
https://dl.acm.org/doi/10.1145/3338906.3342489
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents