More Web Proxy on the site http://driver.im/

research-article

Security Notifications in Static Analysis Tools: Developers’ Attitudes, Comprehension, and Ability to Act on Them

Authors:

Mohammad Tahaei,

Konstantin (Kosta) Beznosov,

Maria K WoltersAuthors Info & Claims

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Article No.: 691, Pages 1 - 17

https://doi.org/10.1145/3411764.3445616

Published: 07 May 2021 Publication History

Abstract

Static analysis tools (SATs) have the potential to assist developers in finding and fixing vulnerabilities in the early stages of software development, requiring them to be able to understand and act on tools’ notifications. To understand how helpful such SAT guidance is to developers, we ran an online experiment (N=132) where participants were shown four vulnerable code samples (SQL injection, hard-coded credentials, encryption, and logging sensitive data) along with SAT guidance, and asked to indicate the appropriate fix. Participants had a positive attitude towards both SAT notifications and particularly liked the example solutions and vulnerable code. Seeing SAT notifications also led to more detailed open-ended answers and slightly improved code correction answers. Still, most SAT (SpotBugs 67%, SonarQube 86%) and Control (96%) participants answered at least one code-correction question incorrectly. Prior software development experience, perceived vulnerability severity, and answer confidence all positively impacted answer accuracy.

References

[1]

Yasemin Acar, Michael Backes, Sascha Fahl, Simson Garfinkel, Doowon Kim, Michelle L Mazurek, and Christian Stransky. 2017. Comparing the Usability of Cryptographic APIs. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 154–171. https://doi.org/10.1109/SP.2017.52

[2]

Yasemin Acar, Michael Backes, Sascha Fahl, Doowon Kim, Michelle L Mazurek, and Christian Stransky. 2016. You Get Where You’re Looking for: The Impact of Information Sources on Code Security. In 2016 IEEE Symposium on Security and Privacy (SP). IEEE, 289–305. https://doi.org/10.1109/SP.2016.25

[3]

Yasemin Acar, Christian Stransky, Dominik Wermke, Michelle L. Mazurek, and Sascha Fahl. 2017. Security Developer Studies with GitHub Users: Exploring a Convenience Sample. In Thirteenth Symposium on Usable Privacy and Security (SOUPS 2017). USENIX Association, Santa Clara, CA, 81–95. https://www.usenix.org/conference/soups2017/technical-sessions/presentation/acar

[4]

Sharmin Afrose, Sazzadur Rahaman, and Danfeng Yao. 2019. CryptoAPI-Bench: A Comprehensive Benchmark on Java Cryptographic API Misuses. In 2019 IEEE Cybersecurity Development (SecDev). IEEE, 49–61. https://doi.org/10.1109/SecDev.2019.00017

[5]

Nathaniel Ayewah, David Hovemeyer, J. David Morgenthaler, John Penix, and William Pugh. 2008. Using Static Analysis to Find Bugs. IEEE Software 25, 5 (Sept. 2008), 22–29. https://doi.org/10.1109/MS.2008.130

Digital Library

[6]

Nathaniel Ayewah and William Pugh. 2008. A Report on a Survey and Study of Static Analysis Users. In Proceedings of the 2008 Workshop on Defects in Large Software Systems (Seattle, Washington) (DEFECTS ’08). Association for Computing Machinery, New York, NY, USA, 1–5. https://doi.org/10.1145/1390817.1390819

Digital Library

[7]

Nathaniel Ayewah and William Pugh. 2010. The Google FindBugs Fixit. In Proceedings of the 19th International Symposium on Software Testing and Analysis (Trento, Italy) (ISSTA ’10). Association for Computing Machinery, New York, NY, USA, 241–252. https://doi.org/10.1145/1831708.1831738

Digital Library

[8]

Titus Barik, Denae Ford, Emerson Murphy-Hill, and Chris Parnin. 2018. How Should Compilers Explain Problems to Developers?. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Lake Buena Vista, FL, USA) (ESEC/FSE 2018). Association for Computing Machinery, New York, NY, USA, 633–643. https://doi.org/10.1145/3236024.3236040

Digital Library

[9]

Titus Barik, Justin Smith, Kevin Lubick, Elisabeth Holmes, Jing Feng, Emerson Murphy-Hill, and Chris Parnin. 2017. Do Developers Read Compiler Error Messages?. In Proceedings of the 39th International Conference on Software Engineering (Buenos Aires, Argentina) (ICSE ’17). IEEE Press, 575–585. https://doi.org/10.1109/ICSE.2017.59

Digital Library

[10]

Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, Articles 67, 1 (2015), 1–48. https://doi.org/10.18637/jss.v067.i01

[11]

Lujo Bauer, Cristian Bravo-Lillo, Lorrie Cranor, and Elli Fragkaki. 2013. Warning Design Guidelines. Technical Report. Carnegie Mellon University. https://www.cylab.cmu.edu/_files/pdfs/tech_reports/CMUCyLab13002.pdf

[12]

Brett A. Becker. 2016. An Effective Approach to Enhancing Compiler Error Messages. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (Memphis, Tennessee, USA) (SIGCSE ’16). Association for Computing Machinery, New York, NY, USA, 126–131. https://doi.org/10.1145/2839509.2844584

Digital Library

[13]

Brett A. Becker, Paul Denny, Raymond Pettit, Durell Bouchard, Dennis J. Bouvier, Brian Harrington, Amir Kamil, Amey Karkare, Chris McDonald, Peter-Michael Osera, and et al.2019. Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research. In Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education (Aberdeen, Scotland UK) (ITiCSE-WGR ’19). Association for Computing Machinery, New York, NY, USA, 177–210. https://doi.org/10.1145/3344429.3372508

Digital Library

[14]

Moritz Beller, Radjino Bholanath, Shane McIntosh, and Andy Zaidman. 2016. Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 1. IEEE, 470–481. https://doi.org/10.1109/SANER.2016.105

[15]

Graham Bleaney and Sinan Cepel. 2020. Pysa: An open source static analysis tool to detect and prevent security issues in Python code. Retrieved August 2020 from https://engineering.fb.com/security/pysa/

[16]

Harry N Boone and Deborah A Boone. 2012. Analyzing Likert Data. Journal of Extension 50, 2 (2012), 1–5. https://www.joe.org/joe/2012april/pdf/JOE_v50_2tt2.pdf

[17]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101. https://doi.org/10.1191/1478088706qp063oa

[18]

Find Security Bugs. 2020. Find Security Bugs. Retrieved June 2020 from https://github.com/find-sec-bugs/find-sec-bugs/tree/master/findsecbugs-samples-java

[19]

James Carifio and Rocco Perla. 2008. Resolving the 50-year debate around using and misusing Likert scales. Medical Education 42, 12 (Dec. 2008), 1150–1152. https://doi.org/10.1111/j.1365-2923.2008.03172.x

[20]

Stephen Cass. 2020. Top Programming Languages 2020. Retrieved July 2020 from https://spectrum.ieee.org/at-work/tech-careers/top-programming-language-2020

[21]

Sen Chen, Lingling Fan, Guozhu Meng, Ting Su, Minhui Xue, Yinxing Xue, Yang Liu, and Lihua Xu. 2020. An Empirical Assessment of Security Risks of Global Android Banking Apps. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 1310–1322. https://doi.org/10.1145/3377811.3380417

Digital Library

[22]

Maria Christakis and Christian Bird. 2016. What Developers Want and Need from Program Analysis: An Empirical Study. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (Singapore, Singapore) (ASE 2016). Association for Computing Machinery, New York, NY, USA, 332–343. https://doi.org/10.1145/2970276.2970347

Digital Library

[23]

TIOBE The Software Quality Company. 2020. TIOBE Index. Retrieved June 2020 from https://www.tiobe.com/tiobe-index/

[24]

The MITRE Corporation. 2020. Common Weakness Enumeration (CWE) Top 25 Most Dangerous Software Errors. Retrieved August 2020 from https://cwe.mitre.org/top25/archive/2019/2019_cwe_top25.html

[25]

Lorrie Faith Cranor. 2008. A Framework for Reasoning about the Human in the Loop. In Proceedings of the 1st Conference on Usability, Psychology, and Security (San Francisco, California) (UPSEC’08). USENIX Association, USA, Article 1, 15 pages. https://www.usenix.org/legacy/events/upsec08/tech/full_papers/cranor/cranor.pdf

Digital Library

[26]

Anastasia Danilova, Alena Naiakshina, and Matthew Smith. 2020. One Size Does Not Fit All: A Grounded Theory and Online Survey Study of Developer Preferences for Security Warning Types. In Proceedings of the 42nd International Conference on Software Engineering(ICSE ’20). Association for Computing Machinery, 13 pages. https://doi.org/10.1145/3377811.3380387

Digital Library

[27]

H T Davies, I K Crombie, and M Tavakoli. 1998. When can odds ratios mislead?BMJ (Clinical research ed.) 316, 7136 (March 1998), 989–991. https://doi.org/10.1136/bmj.316.7136.989 Publisher: British Medical Journal.

[28]

Pieter De Cremer, Nathan Desmet, Matias Madou, and Bjorn De Sutter. 2020. Sensei: Enforcing secure coding guidelines in the integrated development environment. Software: Practice and Experience 50, 9 (2020), 1682–1718. https://doi.org/10.1002/spe.2844

[29]

CVE Details. 2019. CVE-2019-17397 : In the DoorDash application through 11.5.2 for Android, the username and password are stored in the log during authentication, and may be available to attackers via logcat.Retrieved June 2020 from https://www.cvedetails.com/cve/CVE-2019-17397/

[30]

CVE Details. 2020. CVE security vulnerabilities related to CWE (Common Weakness Enumeration) 532. Retrieved June 2020 from https://www.cvedetails.com/vulnerability-list/cweid-532/vulnerabilities.html

[31]

Dino Distefano, Manuel Fähndrich, Francesco Logozzo, and Peter W. O’Hearn. 2019. Scaling Static Analyses at Facebook. Commun. ACM 62, 8 (July 2019), 62–70. https://doi.org/10.1145/3338112

Digital Library

[32]

Sascha Fahl, Marian Harbach, Henning Perl, Markus Koetter, and Matthew Smith. 2013. Rethinking SSL Development in an Appified World. In Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security (Berlin, Germany) (CCS ’13). Association for Computing Machinery, New York, NY, USA, 49–60. https://doi.org/10.1145/2508859.2516655

Digital Library

[33]

Felix Fischer, Konstantin Böttinger, Huang Xiao, Christian Stransky, Yasemin Acar, Michael Backes, and Sascha Fahl. 2017. Stack Overflow Considered Harmful? The Impact of Copy Paste on Android Application Security. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 121–136. https://doi.org/10.1109/SP.2017.31

[34]

Andrew Gelman and Jennifer Hill. 2007. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, Cambridge, UK.

[35]

Andrew Gelman, Masanao Yajima Yu-Sung Su, Jennifer Hill, Maria Grazia Pittau, Jouni Kerman, Tian Zheng, and Vincent Dorie. 2020. arm: Data Analysis Using Regression and Multilevel/Hierarchical Models. Retrieved December 2020 from https://cran.r-project.org/package=arm

[36]

Martin Georgiev, Subodh Iyengar, Suman Jana, Rishita Anubhai, Dan Boneh, and Vitaly Shmatikov. 2012. The Most Dangerous Code in the World: Validating SSL Certificates in Non-Browser Software. In Proceedings of the 2012 ACM Conference on Computer and Communications Security (Raleigh, North Carolina, USA) (CCS ’12). Association for Computing Machinery, New York, NY, USA, 38–49. https://doi.org/10.1145/2382196.2382204

Digital Library

[37]

GitHub. 2019. The State of the Octoverse. Retrieved August 2020 from https://octoverse.github.com/

[38]

Google. 2020. Google Diversity Annual Report. Retrieved August 2020 from https://diversity.google/annual-report/

[39]

Peter Leo Gorski, Yasemin Acar, Luigi Lo Iacono, and Sascha Fahl. 2020. Listen to Developers! A Participatory Design Study on Security Warnings for Cryptographic APIs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376142

Digital Library

[40]

Peter Leo Gorski, Luigi Lo Iacono, Dominik Wermke, Christian Stransky, Sebastian Möller, Yasemin Acar, and Sascha Fahl. 2018. Developers Deserve Security Warnings, Too: On the Effect of Integrated Security Advice on Cryptographic API Misuse. In Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018). USENIX Association, Baltimore, MD, 265–281. https://www.usenix.org/conference/soups2018/presentation/gorski

Digital Library

[41]

Matthew Green and Matthew Smith. 2016. Developers are Not the Enemy!: The Need for Usable Security APIs. IEEE Security & Privacy 14, 05 (Sept. 2016), 40–46. https://doi.org/10.1109/MSP.2016.111

Digital Library

[42]

UCLA: Statistical Consulting Group. 2020. Mixed Effects Logistic Regression. Retrieved December 2020 from https://stats.idre.ucla.edu/stata/dae/mixed-effects-logistic-regression/

[43]

Andrew Habib and Michael Pradel. 2018. How Many of All Bugs Do We Find? A Study of Static Bug Detectors. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (Montpellier, France) (ASE 2018). Association for Computing Machinery, New York, NY, USA, 317–328. https://doi.org/10.1145/3238147.3238213

Digital Library

[44]

Julie M. Haney, Mary Theofanos, Yasemin Acar, and Sandra Spickard Prettyman. 2018. “We make it a big deal in the company”: Security Mindsets in Organizations that Develop Cryptographic Products. In Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018). USENIX Association, Baltimore, MD, 357–373. https://www.usenix.org/conference/soups2018/presentation/haney-mindsets

[45]

Michael Huth and Flemming Nielson. 2019. Static Analysis for Proactive Security. In Computing and Software Science: State of the Art and Perspectives. Springer International Publishing, Cham, 374–392. https://doi.org/10.1007/978-3-319-91908-9_19

Digital Library

[46]

Nasif Imtiaz, Brendan Murphy, and Laurie Williams. 2019. How Do Developers Act on Static Analysis Alerts? An Empirical Study of Coverity Usage. In 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 323–333. https://doi.org/10.1109/ISSRE.2019.00040

[47]

Nasif Imtiaz, Akond Rahman, Effat Farhana, and Laurie Williams. 2019. Challenges with Responding to Static Analysis Tool Alerts. In Proceedings of the 16th International Conference on Mining Software Repositories (Montreal, Quebec, Canada) (MSR ’19). IEEE Press, 245–249. https://doi.org/10.1109/MSR.2019.00049

Digital Library

[48]

Nasif Imtiaz and Laurie Williams. 2019. A Synopsis of Static Analysis Alerts on Open Source Software. In Proceedings of the 6th Annual Symposium on Hot Topics in the Science of Security (Nashville, Tennessee, USA) (HotSoS ’19). Association for Computing Machinery, New York, NY, USA, Article 12, 3 pages. https://doi.org/10.1145/3314058.3317295

Digital Library

[49]

Mazharul Islam, Sazzadur Rahaman, Na Meng, Behnaz Hassanshahi, Padmanabhan Krishnan, and Danfeng (Daphne) Yao. 2020. Coding Practices and Recommendations of Spring Security for Enterprise Applications. In 2020 IEEE Secure Development (SecDev). IEEE, 49–57. https://doi.org/10.1109/SecDev45635.2020.00024

[50]

Shubham Jain, Janne Lindqvist, 2014. Should I protect you? Understanding developers’ behavior to privacy-preserving APIs. In Workshop on Usable Security (USEC’14). Internet Society, 10 pages. https://doi.org/10.14722/usec.2014.23045

[51]

JetBrains. 2020. The State of Developer Ecosystem. Retrieved August 2020 from https://www.jetbrains.com/lp/devecosystem-2020/

[52]

Ling Jin, Boyuan He, Guangyao Weng, Haitao Xu, Yan Chen, and Guanyu Guo. 2019. MAdLens: Investigating into Android In-App Ad Practice at API Granularity. IEEE Transactions on Mobile Computing PP (2019), 18 pages. https://doi.org/10.1109/TMC.2019.2953609

[53]

Brittany Johnson, Rahul Pandita, Justin Smith, Denae Ford, Sarah Elder, Emerson Murphy-Hill, Sarah Heckman, and Caitlin Sadowski. 2016. A Cross-Tool Communication Study on Program Analysis Tool Notifications. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (Seattle, WA, USA) (FSE 2016). Association for Computing Machinery, New York, NY, USA, 73–84. https://doi.org/10.1145/2950290.2950304

Digital Library

[54]

Brittany Johnson, Yoonki Song, Emerson Murphy-Hill, and Robert Bowdidge. 2013. Why Don’t Software Developers Use Static Analysis Tools to Find Bugs?. In Proceedings of the 2013 International Conference on Software Engineering (San Francisco, CA, USA) (ICSE ’13). IEEE Press, 672–681. https://doi.org/10.1109/ICSE.2013.6606613

[55]

John Kelsey, Bruce Schneier, and David Wagner. 1997. Related-key cryptanalysis of 3-WAY, Biham-DES,CAST, DES-X, NewDES, RC2, and TEA. In Information and Communications Security, Yongfei Han, Tatsuaki Okamoto, and Sihan Qing(Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 233–246. https://doi.org/10.1007/BFb0028479

[56]

Jae-on Kim and Charles Mueller. 2020. Factor Analysis. SAGE Publications, Inc, Thousand Oaks, California. https://doi.org/10.4135/9781412984256

[57]

Stefan Krüger, Sarah Nadi, Michael Reif, Karim Ali, Mira Mezini, Eric Bodden, Florian Göpfert, Felix Günther, Christian Weinert, Daniel Demmler, and Ram Kamath. 2017. CogniCrypt: Supporting Developers in Using Cryptography. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering(Urbana-Champaign, IL, USA) (ASE 2017). IEEE Press, 931–936. https://doi.org/10.1109/ASE.2017.8115707

[58]

James A. Kupsch, Elisa Heymann, Barton Miller, and Vamshi Basupalli. 2017. Bad and good news about using software assurance tools. Software: Practice and Experience 47, 1 (2017), 143–156. https://doi.org/10.1002/spe.2401

Digital Library

[59]

Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Chapter 8 - Interviews and focus groups. In Research Methods in Human Computer Interaction (second edition ed.), Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser (Eds.). Morgan Kaufmann, Boston, 187–228. https://doi.org/10.1016/B978-0-12-805390-4.00008-X

[60]

Li Li, Tegawendé F. Bissyandé, Mike Papadakis, Siegfried Rasthofer, Alexandre Bartel, Damien Octeau, Jacques Klein, and Le Traon. 2017. Static analysis of android apps: A systematic literature review. Information and Software Technology 88 (2017), 67–95. https://doi.org/10.1016/j.infsof.2017.04.001

Digital Library

[61]

Na Meng, Stefan Nagy, Danfeng (Daphne) Yao, Wenjie Zhuang, and Gustavo Arango Argoty. 2018. Secure Coding Practices in Java: Challenges and Vulnerabilities. In Proceedings of the 40th International Conference on Software Engineering (Gothenburg, Sweden) (ICSE ’18). Association for Computing Machinery, New York, NY, USA, 372–383. https://doi.org/10.1145/3180155.3180201

Digital Library

[62]

Kai Mindermann, Philipp Keck, and Stefan Wagner. 2018. How Usable Are Rust Cryptography APIs?. In 2018 IEEE International Conference on Software Quality, Reliability and Security (QRS). IEEE, 143–154. https://doi.org/10.1109/qrs.2018.00028

[63]

Kai Mindermann and Stefan Wagner. 2018. Usability and Security Effects of Code Examples on Crypto APIs. In 2018 16th Annual Conference on Privacy, Security and Trust (PST). IEEE, 1–2. https://doi.org/10.1109/PST.2018.8514203

[64]

Kai Mindermann and Stefan Wagner. 2020. Fluid Intelligence Doesn’t Matter! Effects of Code Examples on the Usability of Crypto APIs. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 306–307. https://doi.org/10.1145/3377812.3390892

Digital Library

[65]

Xenia Mountrouidou, David Vosen, Chadi Kari, Mohammad Q. Azhar, Sajal Bhatia, Greg Gagne, Joseph Maguire, Liviana Tudor, and Timothy T. Yuen. 2019. Securing the Human: A Review of Literature on Broadening Diversity in Cybersecurity Education. In Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education (Aberdeen, Scotland UK) (ITiCSE-WGR ’19). Association for Computing Machinery, New York, NY, USA, 157–176. https://doi.org/10.1145/3344429.3372507

Digital Library

[66]

Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden. 2016. Jumping through Hoops: Why Do Java Developers Struggle with Cryptography APIs?. In Proceedings of the 38th International Conference on Software Engineering (Austin, Texas) (ICSE ’16). Association for Computing Machinery, New York, NY, USA, 935–946. https://doi.org/10.1145/2884781.2884790

Digital Library

[67]

Nico JD Nagelkerke 1991. A note on a general definition of the coefficient of determination. Biometrika 78, 3 (09 1991), 691–692. https://doi.org/10.1093/biomet/78.3.691

[68]

Alena Naiakshina, Anastasia Danilova, Eva Gerlitz, and Matthew Smith. 2020. On Conducting Security Developer Studies with CS Students: Examining a Password-Storage Study with CS Students, Freelancers, and Company Developers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376791

Digital Library

[69]

Alena Naiakshina, Anastasia Danilova, Eva Gerlitz, Emanuel von Zezschwitz, and Matthew Smith. 2019. “If You Want, I Can Store the Encrypted Password”: A Password-Storage Field Study with Freelance Developers. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland UK) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300370

Digital Library

[70]

Alena Naiakshina, Anastasia Danilova, Christian Tiefenau, Marco Herzog, Sergej Dechand, and Matthew Smith. 2017. Why Do Developers Get Password Storage Wrong? A Qualitative Usability Study. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (Dallas, Texas, USA) (CCS ’17). Association for Computing Machinery, New York, NY, USA, 311–328. https://doi.org/10.1145/3133956.3134082

Digital Library

[71]

Duc Cuong Nguyen, Dominik Wermke, Yasemin Acar, Michael Backes, Charles Weir, and Sascha Fahl. 2017. A Stitch in Time: Supporting Android Developers in Writing Secure Code. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (Dallas, Texas, USA) (CCS ’17). ACM, New York, NY, USA, 1065–1077. https://doi.org/10.1145/3133956.3133977

Digital Library

[72]

Lisa Nguyen Quang Do, James Wright, and Karim Ali. 2020. Why Do Software Developers Use Static Analysis Tools? A User-Centered Study of Developer Needs and Motivations. IEEE Transactions on Software Engineering PP (2020), 13 pages. https://doi.org/10.1109/TSE.2020.3004525

[73]

National Institute of Standards and Technology (NIST). 2017. Software Assurance Reference Dataset. Retrieved June 2020 from https://samate.nist.gov/SARD/testsuite.php

[74]

Daniela Oliveira, Marissa Rosenthal, Nicole Morin, Kuo-Chuan Yeh, Justin Cappos, and Yanyan Zhuang. 2014. It’s the Psychology Stupid: How Heuristics Explain Software Vulnerabilities and How Priming Can Illuminate Developer’s Blind Spots. In Proceedings of the 30th Annual Computer Security Applications Conference (New Orleans, Louisiana, USA) (ACSAC ’14). Association for Computing Machinery, New York, NY, USA, 296–305. https://doi.org/10.1145/2664243.2664254

Digital Library

[75]

Marten Oltrogge, Erik Derr, Christian Stransky, Yasemin Acar, Sascha Fahl, Christian Rossow, Giancarlo Pellegrino, Sven Bugiel, and Michael Backes. 2018. The Rise of the Citizen Developer: Assessing the Security Impact of Online App Generators. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 634–647. https://doi.org/10.1109/SP.2018.00005

[76]

Stack Overflow. 2020. Developer Survey Results. Retrieved August 2020 from https://insights.stackoverflow.com/survey/2020

[77]

OWASP. 2017. The Ten Most Critical Web Application Security Risks. Retrieved August 2020 from https://owasp.org/www-project-top-ten

[78]

Eyal Peer, Laura Brandimarte, Sonam Samat, and Alessandro Acquisti. 2017. Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology 70 (2017), 153–163. https://doi.org/10.1016/j.jesp.2017.01.006

[79]

Sazzadur Rahaman, Ya Xiao, Sharmin Afrose, Fahad Shaon, Ke Tian, Miles Frantz, Murat Kantarcioglu, and Danfeng (Daphne) Yao. 2019. CryptoGuard: High Precision Detection of Cryptographic Vulnerabilities in Massive-sized Java Projects. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (London, United Kingdom) (CCS ’19). ACM, New York, NY, USA, 2455–2472. https://doi.org/10.1145/3319535.3345659

Digital Library

[80]

Akond Rahman, Chris Parnin, and Laurie Williams. 2019. The Seven Sins: Security Smells in Infrastructure as Code Scripts. In Proceedings of the 41st International Conference on Software Engineering (Montreal, Quebec, Canada) (ICSE ’19). IEEE Press, 164–175. https://doi.org/10.1109/ICSE.2019.00033

Digital Library

[81]

Zachary Reynolds, Abhinandan Jayanth, Ugur Koc, Adam Porter, Rajeev Raje, and James Hill. 2017. Identifying and Documenting False Positive Patterns Generated by Static Code Analysis Tools. In 2017 IEEE/ACM 4th International Workshop on Software Engineering Research and Industrial Practice (SER IP). IEEE, 55–61. https://doi.org/10.1109/SER-IP.2017.20

Digital Library

[82]

Caitlin Sadowski, Edward Aftandilian, Alex Eagle, Liam Miller-Cushon, and Ciera Jaspan. 2018. Lessons from Building Static Analysis Tools at Google. Commun. ACM 61, 4 (March 2018), 58–66. https://doi.org/10.1145/3188720

Digital Library

[83]

Caitlin Sadowski, Jeffrey van Gogh, Ciera Jaspan, Emma Soderberg, and Collin Winter. 2015. Tricorder: Building a Program Analysis Ecosystem. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. IEEE, Florence, Italy, 598–608. https://doi.org/10.1109/ICSE.2015.76

[84]

Neil Salkind. 2020. Encyclopedia of Research Design. SAGE Publications, Inc. https://doi.org/10.4135/9781412961288

[85]

Ben Shneiderman. 1982. Designing computer system messages. Commun. ACM 25, 9 (1982), 610–611. https://doi.org/10.1145/358628.358639

Digital Library

[86]

Miltiadis Siavvas, Erol Gelenbe, Dionysios Kehagias, and Dimitrios Tzovaras. 2018. Static Analysis-Based Approaches for Secure Software Development. In Security in Computer and Information Sciences, Erol Gelenbe, Paolo Campegiani, Tadeusz Czachórski, Sokratis K. Katsikas, Ioannis Komnios, Luigi Romano, and Dimitrios Tzovaras(Eds.). Springer International Publishing, Cham, 142–157. https://doi.org/10.1007/978-3-319-95189-8_13

[87]

Programming skills. 2020. Free Core Java Online Practice Test and Preparation for Exam. Retrieved June 2020 from https://www.pskills.org/corejava.jsp

[88]

Justin Smith, Lisa Nguyen Quang Do, and Emerson Rex Murphy-Hill. 2020. Why Can’t Johnny Fix Vulnerabilities: A Usability Evaluation of Static Analysis Tools for Security. In Sixteenth Symposium on Usable Privacy and Security (SOUPS 2020). USENIX Association, 221–238. https://www.usenix.org/conference/soups2020/presentation/smith

[89]

Justin Smith, Brittany Johnson, Emerson Murphy-Hill, Bill Chu, and Heather Richter Lipford. 2015. Questions Developers Ask While Diagnosing Potential Security Vulnerabilities with Static Analysis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 248–259. https://doi.org/10.1145/2786805.2786812

Digital Library

[90]

Justin Smith, Brittany Johnson, Emerson Murphy-Hill, Bei-Tseng Chu, and Heather Richter. 2019. How Developers Diagnose Potential Security Vulnerabilities with a Static Analysis Tool. IEEE Transactions on Software Engineering 45, 9 (Sept. 2019), 877–897. https://doi.org/10.1109/TSE.2018.2810116

Digital Library

[91]

Justin Smith, Chris Theisen, and Titus Barik. 2020. A Case Study of Software Security Red Teams at Microsoft. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE Computer Society, Los Alamitos, CA, USA, 1–10. https://doi.org/10.1109/VL/HCC50065.2020.9127203

[92]

Mohammad Tahaei and Kami Vaniea. 2019. A Survey on Developer-Centred Security. In 2019 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). IEEE, 129–138. https://doi.org/10.1109/EuroSPW.2019.00021

[93]

Tyler W. Thomas, Heather Lipford, Bill Chu, Justin Smith, and Emerson Murphy-Hill. 2016. What Questions Remain? An Examination of How Developers Understand an Interactive Static Analysis Tool. In Twelfth Symposium on Usable Privacy and Security (SOUPS 2016). USENIX Association, Denver, CO, 7 pages. https://www.usenix.org/system/files/conference/soups2015/wsiw16_paper_thomas.pdf

[94]

V. Javier Traver. 2010. On Compiler Error Messages: What They Say and What They Mean. Adv. in Hum.-Comp. Int. 2010, Article 3 (Jan. 2010), 26 pages. https://doi.org/10.1155/2010/602570

[95]

Omer Tripp, Salvatore Guarnieri, Marco Pistoia, and Aleksandr Aravkin. 2014. ALETHEIA: Improving the Usability of Static Security Analysis. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security (Scottsdale, Arizona, USA) (CCS ’14). Association for Computing Machinery, New York, NY, USA, 762–774. https://doi.org/10.1145/2660267.2660339

Digital Library

[96]

Martin Ukrop, Lydia Kraus, Vashek Matyas, and Heider Ahmad Mutleq Wahsheh. 2019. Will You Trust This TLS Certificate? Perceptions of People Working in IT. In Proceedings of the 35th Annual Computer Security Applications Conference (San Juan, Puerto Rico) (ACSAC ’19). Association for Computing Machinery, New York, NY, USA, 718–731. https://doi.org/10.1145/3359789.3359800

Digital Library

[97]

Dirk van der Linden, Pauline Anthonysamy, Bashar Nuseibeh, Thein Than Tun, Marian Petre, Mark Levine, John Towse, and Awais Rashid. 2020. SchröDinger’s Security: Opening the Box on App Developers’ Security Rationale. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 149–160. https://doi.org/10.1145/3377811.3380394

Digital Library

[98]

Carmine Vassallo, Sebastiano Panichella, Fabio Palomba, Sebastian Proksch, Harald C. Gall, and Andy Zaidman. 2020. How developers engage with static analysis tools in different contexts. Empirical Software Engineering 25, 2 (March 2020), 1419–1457. https://doi.org/10.1007/s10664-019-09750-5

[99]

Carmine Vassallo, Sebastiano Panichella, Fabio Palomba, Sebastian Proksch, Andy Zaidman, and Harald C. Gall. 2018. Context is king: The developer perspective on the usage of static analysis tools. In 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER) (Campobasso). IEEE, 38–49. https://doi.org/10.1109/SANER.2018.8330195

[100]

Daniel Votipka, Desiree Abrokwa, and Michelle L. Mazurek. 2020. Building and Validating a Scale for Secure Software Development Self-Efficacy. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–20. https://doi.org/10.1145/3313831.3376754

Digital Library

[101]

Charles Weir, Ingolf Becker, James Noble, Lynne Blair, M. Angela Sasse, and Awais Rashid. 2020. Interventions for long-term software security: Creating a lightweight program of assurance techniques for developers. Software: Practice and Experience 50, 3 (2020), 275–298. https://doi.org/10.1002/spe.2774

[102]

William R. Nichols and Thomas Scanlon. 2018. DoD Developer’s Guidebook for Software Assurance. Technical Report. Software Engineering Institute - Carnegie Mellon University. 111pages. https://resources.sei.cmu.edu/asset_files/SpecialReport/2018_003_001_538761.pdf

[103]

Brett Williams, Andrys Onsman, and Ted Brown. 2010. Exploratory factor analysis: A five-step guide for novices. Australasian Journal of Paramedicine 8, 3 (2010), 1–13. https://doi.org/10.33151/ajp.8.3.93

[104]

Michael S Wogalter. 2019. Communication-Human Information Processing (C-HIP) Model in Forensic Warning Analysis. In Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018), Sebastiano Bagnara, Riccardo Tartaglia, Sara Albolino, Thomas Alexander, and Yushi Fujita(Eds.). Springer International Publishing, Cham, 761–769. https://doi.org/10.1007/978-3-319-96080-7_92

[105]

Fiorella Zampetti, Simone Scalabrino, Rocco Oliveto, Gerardo Canfora, and Massimiliano Di Penta. 2017. How Open Source Projects Use Static Code Analysis Tools in Continuous Integration Pipelines. In Proceedings of the 14th International Conference on Mining Software Repositories (Buenos Aires, Argentina) (MSR ’17). IEEE Press, 334–344. https://doi.org/10.1109/MSR.2017.2

Digital Library

[106]

Sebastian Zimmeck, Ziqi Wang, Lieyong Zou, Roger Iyengar, Bin Liu, Florian Schaub, Shomir Wilson, Norman M Sadeh, Steven M Bellovin, and Joel R Reidenberg. 2017. Automated Analysis of Privacy Requirements for Mobile Apps. In NDSS. Internet Society, 15 pages. https://doi.org/10.14722/ndss.2017.23034

Cited By

Chen LZhang SQin YCao KDu TDai Q(2025)Software defect prediction based on support vector machine optimized by reverse differential chimp optimization algorithmInternational Journal of Data Science and Analytics10.1007/s41060-025-00726-xOnline publication date: 4-Feb-2025
https://doi.org/10.1007/s41060-025-00726-x
Lee HGao LYang SForlizzi JDas SBalzarotti DXu W(2024)"I don't know if we're doing good. I don't know if we're doing bad": investigating how practitioners scope, motivate, and conduct privacy work when developing AI productsProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699173(4873-4890)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699173
Jawad AAssal HJaskolka J(2024)"I'm Getting Information that I Can Act on Now": Exploring the Level of Actionable Information in Tool-generated Threat ReportsProceedings of the 2024 European Symposium on Usable Security10.1145/3688459.3688467(172-186)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3688459.3688467
Show More Cited By

Index Terms

Security Notifications in Static Analysis Tools: Developers’ Attitudes, Comprehension, and Ability to Act on Them

Index terms have been assigned to the content through auto-classification.

Recommendations

Comparison of Static Analysis Tools for Finding Concurrency Bugs
ISSREW '12: Proceedings of the 2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops

This paper highlights the issues of detecting Java concurrency bugs using static code analysis tools. Concurrency bugs are often hard to find because of interleaving threads and there is need to use static analysis tools to detect the concurrency bugs. ...
Can static analysis tools find more defects?: A qualitative study of design rule violations found by code review
Abstract
Static analysis tools find defects in code, checking code against rules to reveal potential defects. Many studies have evaluated these tools by measuring their ability to detect known defects in code. But these studies measure the current state of ...
Effect of static analysis tools on software security: preliminary investigation
QoP '07: Proceedings of the 2007 ACM workshop on Quality of protection

Static analysis tools can handle large-scale software and find thousands of defects. But do they improve software security? We evaluate the effect of static analysis tool use on software security in open source projects. We measure security by ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

10862 pages

ISBN:9781450380966

DOI:10.1145/3411764

General Chairs:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Program Chairs:
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan
,
Publications Chairs:
Pernille Bjørn
University of Copenhagen, Denmark
,
Steven Drucker
Microsoft Research, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
775
Total Downloads

Downloads (Last 12 months)170
Downloads (Last 6 weeks)58

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen LZhang SQin YCao KDu TDai Q(2025)Software defect prediction based on support vector machine optimized by reverse differential chimp optimization algorithmInternational Journal of Data Science and Analytics10.1007/s41060-025-00726-xOnline publication date: 4-Feb-2025
https://doi.org/10.1007/s41060-025-00726-x
Lee HGao LYang SForlizzi JDas SBalzarotti DXu W(2024)"I don't know if we're doing good. I don't know if we're doing bad": investigating how practitioners scope, motivate, and conduct privacy work when developing AI productsProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699173(4873-4890)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699173
Jawad AAssal HJaskolka J(2024)"I'm Getting Information that I Can Act on Now": Exploring the Level of Actionable Information in Tool-generated Threat ReportsProceedings of the 2024 European Symposium on Usable Security10.1145/3688459.3688467(172-186)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3688459.3688467
Kim TPark SKim H(2023)Why Johnny Can’t Use Secure Docker Images: Investigating the Usability Challenges in Using Docker Image Vulnerability Scanners through Heuristic EvaluationProceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3607199.3607244(669-685)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3607199.3607244
Senanayake JKalutarage HAl-Kadri MPetrovski APiras L(2023)Android Source Code Vulnerability Detection: A Systematic Literature ReviewACM Computing Surveys10.1145/355697455:9(1-37)Online publication date: 16-Jan-2023
https://dl.acm.org/doi/10.1145/3556974
Plöger SMeier MSmith M(2023)A Usability Evaluation of AFL and libFuzzer with CS StudentsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581178(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581178
Kühtreiber PPak VReinhardt D(2023)“A method like this would be overkill”: Developers’ Perceived Issues with Privacy-preserving Computation Methods2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom)10.1109/TrustCom60117.2023.00145(1041-1048)Online publication date: 1-Nov-2023
https://doi.org/10.1109/TrustCom60117.2023.00145
Ryan IStol KRoedig U(2023)The State of Secure Coding Practice: Small Organisations and “Lone, Rogue Coders”2023 IEEE/ACM 4th International Workshop on Engineering and Cybersecurity of Critical Systems (EnCyCriS)10.1109/EnCyCriS59249.2023.00010(37-44)Online publication date: May-2023
https://doi.org/10.1109/EnCyCriS59249.2023.00010
Kang HAw KLo DDwyer MDamian DZeller A(2022)Detecting false alarms from automatic static analysis toolsProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510214(698-709)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510214
Tahaei MVaniea K(2022)Recruiting Participants With Programming Skills: A Comparison of Four Crowdsourcing Platforms and a CS Student Mailing ListProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501957(1-15)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501957
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten