Evaluation of Free and Open Source Tools for Automated Software Composition Analysis
Authors: 
Laura Bottner, Artur Hermann, Jeremias Eppler, 
Thomas Thüm, 
Frank KarglAuthors Info & Claims
Article No.: 3, Pages 1 - 11
Abstract
Vulnerable or malicious third-party components introduce vulnerabilities into the software supply chain. Software Composition Analysis (SCA) is a method to identify direct and transitive dependencies in software projects and assess their security risks and vulnerabilities.
In this paper, we investigate two open source SCA tools, Eclipse Steady (ES) and OWASP Dependency Check (ODC), with respect to vulnerability detection in Java projects. Both tools use different vulnerability detection methods. ES implements a code-centric and ODC a metadata-based approach. Our study reveals that both tools suffer from false positives. Furthermore, we discover that the success of the vulnerability detection depends on the underlying vulnerability database. Especially ES suffered from false negatives because of the insufficient vulnerability information in the database.
While code-centric and metadata-based approaches offer significant potential, they also come with their respective downsides. We propose a hybrid approach assuming that combining both detection methods will lead to less false negatives and false positives.
References
[1]
[n. d.]. NVD - CVE-2021-44228. https://nvd.nist.gov/vuln/detail/CVE-2021-44228#match-9066512
[2]
2017. Clemson Vehicular Electronics Laboratory: Automotive Electronic Systems. http://web.archive.org/web/20171120173150http://www.cvel.clemson.edu/auto/systems/auto-systems.html
[3]
Vard Antinyan. 2020. Revealing the complexity of automotive software. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, Virtual Event USA, 1525–1528. https://doi.org/10.1145/3368089.3417038
[4]
Mircea Cadariu, Eric Bouwers, Joost Visser, and Arie van Deursen. 2015. Tracking known security vulnerabilities in proprietary software systems. In 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, 516–519.
[5]
Brant A Cheikes, David Waltermire, and Karen Scarfone. 2011. Common platform enumeration :: naming specification version 2.3. Technical Report NIST IR 7695. National Institute of Standards and Technology, Gaithersburg, MD. NIST IR 7695 pages. https://doi.org/10.6028/NIST.IR.7695 Edition: 0.
[6]
Tamal Das. 2022. What CPU Does a Car ECU Run On?https://www.makeuseof.com/cpu-for-car-ecu/ Section: Technology Explained.
[7]
Sébastien Dudek. 2021. Examining Log4j Vulnerabilities in Connected Cars and Charging Stations. https://www.trendmicro.com/en_us/research/21/l/examining-log4j-vulnerabilities-in-connected-cars.html Section: research.
[8]
Christof Ebert and John Favaro. 2017. Automotive Software. IEEE Software 34, 3 (May 2017), 33–39. https://doi.org/10.1109/MS.2017.82 Conference Name: IEEE Software.
[9]
Ian Foster, Andrew Prudhomme, Karl Koscher, and Stefan Savage. 2015. Fast and vulnerable: a story of telematic failures. In Proceedings of the 9th USENIX Conference on Offensive Technologies(WOOT’15). USENIX Association, USA, 15.
[10]
Eclipse Foundation. [n. d.]. Eclipse Steady Analysis Manual. https://eclipse.github.io/steady/user/manuals/analysis/
[11]
OWASP Foundation. 2023. Software Component Verification Standard Measure and Improve Software Supply Chain Assurance. https://scvs.owasp.org/
[12]
The Linux Foundation. 2023. Safeguarding artifact integrity across any software supply chain. https://slsa.dev/spec/v0.1/threats
[13]
Andy Greenberg. 2015. Hackers Remotely Kill a Jeep on the Highway—With Me in It. Wired (July 2015). https://www.wired.com/2015/07/hackers-remotely-kill-jeep-highway/
[14]
Joseph Hejderup, Moritz Beller, Konstantinos Triantafyllou, and Georgios Gousios. 2021. Präzi: From Package-based to Call-based Dependency Networks. CoRR abs/2101.09563 (2021). arXiv:2101.09563https://arxiv.org/abs/2101.09563
[15]
Freddie Holmes. 2018. Auto industry’s thirst for software is quenched by open source. https://www.automotiveworld.com/articles/auto-industrys-thirst-for-software-is-quenched-by-open-source/
[16]
Nasif Imtiaz, Seaver Thorn, and Laurie Williams. 2021. A comparative study of vulnerability reporting by software composition analysis tools. In Proceedings of the 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1–11.
[17]
Jeremy Long and Seth Jackson. 2017. Architecture. https://github.com/jeremylong/DependencyCheck/wiki/Architecture
[18]
Jeremy Long and JoyChou. 2017. How does it work?https://github.com/jeremylong/DependencyCheck/wiki/How-does-it-work%3F
[19]
Jürgen Mössinger. 2010. Software in Automotive Systems. IEEE Software 27, 2 (March 2010), 92–94. https://doi.org/10.1109/MS.2010.55
[20]
Sen Nie, Ling Liu, Yuefeng Du, and Wenkai Zhang. 2018. OVER-THE-AIR: HOW WE REMOTELY COMPROMISED THE GATEWAY, BCM, AND AUTOPILOT ECUS OF TESLA CARS. (Aug. 2018). http://i.blackhat.com/us-18/Thu-August-9/us-18-Liu-Over-The-Air-How-We-Remotely-Compromised-The-Gateway-Bcm-And-Autopilot-Ecus-Of-Tesla-Cars-wp.pdf
[21]
Emily Olin. 2020. Subaru Adopts AGL Software for Infotainment on New 2020 Subaru Outback and Subaru Legacy. https://www.automotivelinux.org/announcements/subaru-outback/
[22]
Paul O’shea. 2017. Automotive electronics: What are they, and how do they differ from "normal" electronics?https://www.powerelectronicsnews.com/automotive-electronics-what-are-they-and-how-do-they-differ-from-normal-electronics/
[23]
Ivan Pashchenko, Henrik Plate, Serena Elisa Ponta, Antonino Sabetta, and Fabio Massacci. 2018. Vulnerable open source dependencies: Counting those that matter. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. 1–10.
[24]
Amrita Pathak. [n. d.]. Software Composition Analysis (SCA): Everything You Need to Know in 2022. https://geekflare.com/software-composition-analysis/
[25]
Etiel Petrinja, Ranga Nambakam, and Alberto Sillitti. 2009. Introducing the OpenSource Maturity Model. In 2009 ICSE Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development. 37–41. https://doi.org/10.1109/FLOSS.2009.5071358
[26]
Serena Elisa Ponta, Henrik Plate, and Antonino Sabetta. 2020. Detection, assessment and mitigation of vulnerabilities in open source dependencies. Empirical Software Engineering 25, 5 (2020), 3175–3215.
[27]
Serena Elisa Ponta, Henrik Plate, Antonino Sabetta, Michele Bezzi, and Cédric Dangremont. 2019. A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). 383–387. https://doi.org/10.1109/MSR.2019.00064 ISSN: 2574-3864.
[28]
Gede Artha Azriadi Prana, Abhishek Sharma, Lwin Khin Shar, Darius Foo, Andrew E Santosa, Asankhaya Sharma, and David Lo. 2021. Out of sight, out of mind? How vulnerable dependencies affect open-source projects. Empirical Software Engineering 26, 4 (2021), 1–34.
[29]
Robert N. Charette. 2009. This Car Runs on Code - IEEE Spectrum. https://spectrum.ieee.org/this-car-runs-on-code
[30]
Sam Curry. 2023. Web Hackers vs. The Auto Industry: Critical Vulnerabilities in Ferrari, BMW, Rolls Royce, Porsche, and More. https://samcurry.net/web-hackers-vs-the-auto-industry/
[31]
Karthik Shanmugam. 2019. Securing Inter-Processor Communication in Automotive ECUs. 2019–26–0363. https://doi.org/10.4271/2019-26-0363
[32]
Synopsys. 2023. Open Source Security and Risk Analysis. Technical Report. https://www.synopsys.com/content/dam/synopsys/sig-assets/reports/rep-ossra-2023.pdf
[33]
Tencent Keen Security Keen Lab. 2021. Tencent Security Keen Lab: Experimental Security Assessment of Mercedes-Benz Cars. http://keenlab.tencent.com/2021/05/12/Tencent-Security-Keen-Lab-Experimental-Security-Assessment-on-Mercedes-Benz-Cars/index.html
[34]
Upstream. 2022. Global Automotive Cybersecurity Report 2022. Technical Report. https://info.upstream.auto/hubfs/Security_Report/Security_Report_2022/Upstream_Security-Global_Automotive_Cybersecurity_Report_2022.pdf
[35]
Chris Valasek and Charlie Miller. 2014. A Survey of Remote Automotive Attack Surfaces. (July 2014). https://ioactive.com/wp-content/uploads/2018/05/IOActive_Remote_Attack_Surfaces.pdf
[36]
Chris Valasek and Charlie Miller. 2015. Remote Exploitation of an Unaltered Passenger Vehicle. (Aug. 2015). https://ioactive.com/wp-content/uploads/2018/05/IOActive_Remote_Car_Hacking-1.pdf
[37]
VicOne. 2022. Steering Clear: VicOne 2022 Automotive Cybersecurity Report. Technical Report. https://vicone.com/files/rpt-automotive-cybersecurity-in-2022.pdf
[38]
Free Wortley, Forrest Allison, and Chris Thompson. 2021. Log4Shell: RCE 0-day exploit found in log4j, a popular Java logging package | LunaTrace. https://www.lunasec.io/docs/blog/log4j-zero-day/
[39]
Boming Xia, Tingting Bi, Zhenchang Xing, Qinghua Lu, and Liming Zhu. 2023. An Empirical Study on Software Bill of Materials: Where We Stand and the Road Ahead. arXiv preprint arXiv:2301.05362 (2023).
[40]
Maria Zhdanova, Julian Urbansky, Anne Hagemeier, Daniel Zelle, Isabelle Herrmann, and Dorian Höffner. 2022. Local Power Grids at Risk – An Experimental and Simulation-based Analysis of Attacks on Vehicle-To-Grid Communication. In Proceedings of the 38th Annual Computer Security Applications Conference(ACSAC ’22). Association for Computing Machinery, New York, NY, USA, 42–55. https://doi.org/10.1145/3564625.3568136
Index Terms
- Evaluation of Free and Open Source Tools for Automated Software Composition Analysis
Recommendations
A comparative study of vulnerability reporting by software composition analysis tools
ESEM '21: Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)Background: Modern software uses many third-party libraries and frameworks as dependencies. Known vulnerabilities in these dependencies are a potential security risk. Software composition analysis (SCA) tools, therefore, are being increasingly adopted ...
Adversarial Analysis of Software Composition Analysis Tools
Information SecurityAbstractWith the widespread use of third-party code in software projects, Software Composition Analysis (SCA) tools emerged in order to help developers and security specialists automate the process of vulnerability detection within dependencies. Among SCA ...
Out of sight, out of mind? How vulnerable dependencies affect open-source projects
AbstractContextSoftware developers often use open-source libraries in their project to improve development speed. However, such libraries may contain security vulnerabilities, and this has resulted in several high-profile incidents in recent years. As ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
December 2023
104 pages
ISBN:9798400704543
DOI:10.1145/3631204
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 December 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 234Total Downloads
- Downloads (Last 12 months)234
- Downloads (Last 6 weeks)16
Reflects downloads up to 30 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format