[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3313831.3376754acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Building and Validating a Scale for Secure Software Development Self-Efficacy

Published: 23 April 2020 Publication History

Abstract

Security is an essential component of the software development lifecycle. Researchers and practitioners have developed educational interventions, guidelines, security analysis tools, and new APIs aimed at improving security. However, measuring any resulting improvement in secure development skill is challenging. As a proxy for skill, we propose to measure self-efficacy, which has been shown to correlate with skill in other contexts. Here, we present a validated scale measuring secure software-development self-efficacy (SSD-SES). We first reviewed popular secure-development frameworks and surveyed 22 secure-development experts to identify 58 unique tasks. Next, we asked 311 developers - over multiple rounds - to rate their skill at each task. We iteratively updated our questions to ensure they were easily understandable, showed adequate variance between participants, and demonstrated reliability. Our final 15-item scale contains two sub-scales measuring belief in ability to perform vulnerability identification and mitigation as well as security communications tasks.

Supplementary Material

ZIP File (pn8409aux.zip)
The auxiliary material contains a single PDF file with multiple appendices. This includes the text of both surveys administered (i.e., to security experts and developers) and additional data not presented in the paper for space considerations. Specifically, the additional data includes the distribution of software development experience across participants assigned each additional scale and the set of initial items (with sources) generated through our review of relevant frameworks.
MP4 File (paper625pv.mp4)
Preview video
MP4 File (a625-votipka-presentation.mp4)

References

[1]
Yasemin Acar, Michael Backes, Sascha Fahl, Simson L. Garfinkel, Doowon Kim, Michelle L. Mazurek, and Christian Stransky. 2017. Comparing the Usability of Cryptographic APIs. In Proceedings of the 38th IEEE Symposium on Security and Privacy (SP '17). IEEE Computer Society, 154--171. http://dblp.uni-trier.de/ db/conf/sp/sp2017.html#Acar0FGKMS17
[2]
Y. Acar, M. Backes, S. Fahl, D. Kim, M. L. Mazurek, and C. Stransky. 2016. You Get Where You're Looking for: The Impact of Information Sources on Code Security. In Proceedings of the 37th IEEE Symposium on Security and Privacy (SP '16). IEEE Computer Society, Los Alamitos, CA, USA, 289--305.
[3]
Yasemin Acar, Christian Stransky, Dominik Wermke, Michelle L. Mazurek, and Sascha Fahl. 2017. Security Developer Studies with GitHub Users: Exploring a Convenience Sample. In Proceedings of the 13th Symposium on Usable Privacy and Security (SOUPS '17). USENIX Association, Santa Clara, CA, 81--95. https://www.usenix.org/conference/soups2017/ technical-sessions/presentation/acar
[4]
Willem A. Arrindell and Jan van der Ende. 1985. An Empirical Test of the Utility of the Observations-To-Variables Ratio in Factor and Components Analysis. Applied Psychological Measurement 9, 2 (1985), 165--178.
[5]
Hala Assal and Sonia Chiasson. 2019. 'Think Secure from the Beginning': A Survey with Software Developers. In Proceedings of the 37th CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 289, 13 pages.
[6]
Andrew R Baggaley. 1983. Deciding on the ratio of number of subjects to number of variables in factor analysis. Multivariate Experimental Clinical Research (1983).
[7]
Richard P. Bagozzi and Youjae Yi. 1988. On the evaluation of structural equation models. Journal of the Academy of Marketing Science 16, 1 (01 Mar 1988), 74--94.
[8]
Rebecca Balebako, Abigail Marsh, Jialiu Lin, Jason I. Hong, and Lorrie Cranor. 2014. The Privacy and Security Behaviors of Smartphone App Developers. (2 2014).
[9]
Sebastian Baltes and Stephan Diehl. 2018. Towards a Theory of Software Development Expertise. In Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018). ACM, NY, NY, USA, 187--200.
[10]
Albert Bandura. 1993. Perceived Self-Efficacy in Cognitive Development and Functioning. Educational Psychologist 28, 2 (1993), 117--148.
[11]
Paul T Barrett and Paul Kline. 1981. The observation to variable ratio in factor analysis. Personality Study & Group Behaviour (1981).
[12]
Peter M Bentler. 1990. Comparative fit indexes in structural models. Psychological bulletin 107, 2 (1990), 238.
[13]
Peter M Bentler and Douglas G Bonett. 1980. Significance tests and goodness of fit in the analysis of covariance structures. Psychological bulletin 88, 3 (1980), 588.
[14]
Kevin Bock, George Hughey, and Dave Levin. 2018. King of the Hill: A Novel Cybersecurity Competition for Teaching Penetration Testing. In Proceedings of the 3rd USENIX Workshop on Advances in Security Education (ASE '18). USENIX Association, Baltimore, MD. https://www.usenix.org/conference/ase18/presentation/bock
[15]
Kenneth A Bollen. 2014. Structural equations with latent variables. Vol. 210. John Wiley & Sons.
[16]
Richard W Brislin. 1980. Cross-cultural research methods. In Environment and culture. Springer, 47--82.
[17]
Jean-Marie Burkhardt, Françoise Détienne, and Susan Wiedenbeck. 2002. Object-Oriented Program Comprehension: Effect of Expertise, Task and Phase. Empirical Software Engineering 7, 2 (01 Jun 2002), 115--156.
[18]
John T. Cacioppo, Richard E. Petty, and Chuan Feng Kao. 1984. The Efficient Assessment of Need for Cognition. Journal of Personality Assessment 48, 3 (1984), 306--307. 16367530.
[19]
A Colin Cameron and Pravin K Trivedi. 2013. Regression analysis of count data. Vol. 53. Cambridge university press.
[20]
Susan G. Campbell, Polly O'Rourke, and Michael F. Bunting. 2015. Identifying Dimensions of Cyber Aptitude: The Design of the Cyber Aptitude and Talent Assessment. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 59, 1 (2015), 721--725.
[21]
Edward G Carmines and John McIver. 1981. Analyzing models with unobserved variables. Social measurement: Current issues 80 (1981).
[22]
Raymond B. Cattell. 1952. Factor analysis: An introduction and manual for the psychologist and social scientist. Harper & Row, New York.
[23]
Center for Cyber Safety and Education. 2017. Global Information Security Workforce Study. Technical Report. Center for Cyber Safety and Education, Clearwater, FL. https://iamcybersafe.org/wp-content/uploads/2017/07/N-America-GISWS-Report.pdf
[24]
Barbara A. Cerny and Henry F. Kaiser. 1977. A Study Of A Measure Of Sampling Adequacy For Factor-Analytic Correlation Matrices. Multivariate Behavioral Research 12, 1 (1977), 43--47. 26804143.
[25]
Pravir Chandra. 2017. Software Assurance Maturity Model. Technical Report. Open Web Application Security Project.
[26]
Wu chang Feng, Robert Liebman, Lois Delcambre, Michael Lupro, Tim Sheard, Scott Britell, and Gerald Recktenwald. 2017. CyberPDX: A Camp for Broadening Participation in Cybersecurity. In Proceedings of the 2nd USENIX Workshop on Advances in Security Education (ASE '17). USENIX Association, Vancouver, BC. https://www.usenix.org/conference/ ase17/workshop-program/presentation/feng
[27]
Peter Chapman, Jonathan Burket, and David Brumley. 2014. PicoCTF: A Game-Based Computer Security Competition for High School Students. In Proceedings of the 1st USENIX Summit on Gaming, Games, and Gamification in Security Education (3GSE '14). USENIX Association, San Diego, CA. http://www.usenix.org/conference/3gse14/summit-program/presentation/chapman
[28]
Kevin Chung and Julian Cohen. 2014. Learning Obstacles in the Capture The Flag Model. In Proceedings of the 1st USENIX Summit on Gaming, Games, and Gamification in Security Education (3GSE '14). USENIX Association, San Diego, CA. https://www.usenix.org/conference/3gse14/ summit-program/presentation/chung
[29]
Jacob Cohen. 2013. Statistical power analysis for the behavioral sciences. Routledge.
[30]
CollabNet. 2019. 13th Annual State of Agile Report. https://www.stateofagile.com/ #ufh-i-521251909--13th-annual-state-of-agile-report/ 473508. (2019).
[31]
Andrew L Comrey. 1973. A first course in factor analysis. Academic Press, New York.
[32]
Andrew L Comrey. 1978. Common methodological problems in factor analytic studies. Journal of consulting and clinical psychology 46, 4 (1978), 648.
[33]
Douglas P Crowne and David Marlowe. 1960. A new scale of social desirability independent of psychopathology. Journal of consulting psychology 24, 4 (1960), 349.
[34]
Erik Derr, Sven Bugiel, Sascha Fahl, Yasemin Acar, and Michael Backes. 2017. Keep Me Updated: An Empirical Study of Third-Party Library Updatability on Android. In Proceedings of the 24th Conference on Computer and Communications Security (CCS '17). ACM, NY, NY, USA, 2187--2200.
[35]
Constanze Dietrich, Katharina Krombholz, Kevin Borgolte, and Tobias Fiebig. 2018. Investigating System Operators' Perspective on Security Misconfigurations. In Proceedings of the 25th ACM Conference on Computer and Communications Security (CCS '18). ACM.
[36]
Wenliang Du. 2011. SEED: hands-on lab exercises for computer security education. IEEE Security & Privacy 9, 5 (2011), 70--73.
[37]
C. Eagle. 2013. Computer Security Competitions: Expanding Educational Outcomes. IEEE Security Privacy 11, 4 (July 2013), 69--71.
[38]
Serge Egelman and Eyal Peer. 2015. Scaling the Security Wall: Developing a Security Behavior Intentions Scale (SeBIS). In Proceedings of the 33rd CHI Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 2873--2882.
[39]
Brian S Everitt and Anders Skrondal. 2010. The Cambridge dictionary of statistics. New York University.
[40]
Leandre R Fabrigar, Duane T Wegener, Robert C MacCallum, and Erin J Strahan. 1999. Evaluating the use of exploratory factor analysis in psychological research. Psychological methods 4, 3 (1999), 272.
[41]
Cori Faklaris, Laura A. Dabbish, and Jason I. Hong. 2019. A Self-Report Measure of End-User Security Attitudes (SA-6). In Proceedings of the 5th Symposium on Usable Privacy and Security (SOUPS '19). USENIX Association, Santa Clara, CA. https://www.usenix.org/conference/soups2019/presentation/faklaris
[42]
The Association for Computing Machinery. 2019. Chapters. https://acm.org/chapters. (2019).
[43]
Claes Fornell and David F. Larcker. 1981. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research 18, 1 (1981), 39--50.
[44]
Gordon Fraser, Alessio Gambi, Marvin Kreis, and José Miguel Rojas. 2019. Gamifying a Software Testing Course with Code Defenders. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19). ACM, NY, NY, USA, 571--577.
[45]
Richard L. Gorsuch. 1978. Factor Analysis (2nd ed.). Erlbaum, Hillsdale, NJ.
[46]
Richard L. Gorsuch. 1988. Exploratory Factor Analysis. Springer US, Boston, MA, 231--258.
[47]
Edward Guadagnoli and Wayne F Velicer. 1988. Relation of sample size to the stability of component patterns. Psychological bulletin 103, 2 (1988), 265.
[48]
Joy Paul Guilford. 1954. Psychometric methods. (1954).
[49]
Joseph F Hair, William C Black, Barry J Babin, Rolph E Anderson, Ronald L Tatham, and others. 2006. Multivariate data analysis. (2006).
[50]
Julie M. Haney and Wayne G. Lutters. 2018. "It's Scary...It's Confusing...It's Dull": How Cybersecurity Advocates Overcome Negative Perceptions of Security. In Proceedings of the 14th Symposium on Usable Privacy and Security (SOUPS '18). USENIX Association, Baltimore, MD, 411--425. https://www.usenix.org/conference/soups2018/ presentation/haney-perceptions
[51]
Julie M. Haney, Mary Theofanos, Yasemin Acar, and Sandra Spickard Prettyman. 2018. "We make it a big deal in the company": Security Mindsets in Organizations that Develop Cryptographic Products. In Proceedings of the 14th Symposium on Usable Privacy and Security (SOUPS '18). USENIX Association, Baltimore, MD, 357--373. https://www.usenix.org/ conference/soups2018/presentation/haney-mindsets
[52]
Mariana Hentea, Harpal S Dhillon, and Manpreet Dhillon. 2006. Towards changes in information security education. Journal of Information Technology Education: Research 5 (2006), 221--233.
[53]
Michael Hilton, Nicholas Nelson, Timothy Tunnell, Darko Marinov, and Danny Dig. 2017. Trade-offs in Continuous Integration: Assurance, Security, and Flexibility. In Proceedings of the 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE '17). ACM, NY, NY, USA, 197--207.
[54]
Timothy R. Hinkin, J. Bruce Tracey, and Cathy A. Enz. 1997. Scale Construction: Developing Reliable and Valid Measurement Instruments. Journal of Hospitality & Tourism Research 21, 1 (1997), 100--120.
[55]
Sture Holm. 1979. A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics 6, 2 (1979), 65--70. http://www.jstor.org/stable/4615733
[56]
John L. Horn. 1965. A rationale and test for the number of factors in factor analysis. Psychometrika 30, 2 (01 Jun 1965), 179--185.
[57]
Gary Hsieh and Rafal Kocielnik. 2016. You Get Who You Pay for: The Impact of Incentives on Participation Bias. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, NY, NY, USA, 823--835.
[58]
Luigi Lo Iacono and Peter Leo Gorski. 2017. I Do and I Understand. Not Yet True for Security APIs. So Sad. In Proceedings of the 2nd European Workshop on Usable Security (EuroUSEC '17). Internet Society. https://doi.org/10.14722/eurousec
[59]
Ge Jin, Manghui Tu, Tae-Hoon Kim, Justin Heffron, and Jonathan White. 2018. Game Based Cybersecurity Training for High School Students. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE '18). ACM, NY, NY, USA, 68--73.
[60]
Brittany Johnson, Yoonki Song, Emerson Murphy-Hill, and Robert Bowdidge. 2013. Why Don't Software Developers Use Static Analysis Tools to Find Bugs?. In Proceedings of the 35th International Conference on Software Engineering (ICSE '13). IEEE Press, 672--681.
[61]
Melanie Jones. 2019. Why cybersecurity education matters. https://www.itproportal.com/features/why-cybersecurity-education-matters/. (2019).
[62]
Karl G Jöreskog and Dag Sörbom. 1993. LISREL 8: Structural equation modeling with the SIMPLIS command language. Scientific Software International.
[63]
Lindsay Kolowich. 2017. The Demographics of Developers Around the World. https://blog.hubspot.com/marketing/developers-demographic-survey. (2017).
[64]
Stefan Krüger, Johannes Späth, Karim Ali, Eric Bodden, and Mira Mezini. 2017. CrySL: Validating Correct Usage of Cryptographic APIs. CoRR abs/1710.00564 (2017). http://arxiv.org/abs/1710.00564
[65]
Joseph T. Kunce, Daniel W. Cook, and Douglas E. Miller. 1975. Random Variables and Correlational Overkill. Educational and Psychological Measurement 35, 3 (1975), 529--534.
[66]
Thomas D. LaToza and Brad A. Myers. 2010. Developers Ask Reachability Questions. In Proceedings of the 32nd International Conference on Software Engineering (ICSE '10). ACM, NY, NY, USA, 185--194.
[67]
Ákos Lédeczi, MiklÓs MarÓti, Hamid Zare, Bernard Yett, Nicole Hutchins, Brian Broll, Péter Völgyesi, Michael B. Smith, Timothy Darrah, Mary Metelko, Xenofon Koutsoukos, and Gautam Biswas. 2019. Teaching Cybersecurity with Networked Robots. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19). ACM, NY, NY, USA, 885--891.
[68]
Timothy C Lethbridge, Jorge Diaz-Herrera, Richard Jr J LeBlanc, and J Barrie Thompson. 2007. Improving software practice through education: Challenges and future trends. In Future of Software Engineering. IEEE Computer Society, 12--28.
[69]
Paul Luo Li, Andrew J. Ko, and Jiamin Zhu. 2015. What Makes a Great Software Engineer?. In Proceedings of the 37th International Conference on Software Engineering (ICSE '15). IEEE Press, Piscataway, NJ, USA, 700--710. http://dl.acm.org/citation.cfm?id=2818754.2818839
[70]
Richard Harold Lindeman. 1980. Introduction to bivariate and multivariate analysis. Technical Report.
[71]
LinkedIn. 2019. LinkedIn. https://linkedin.com/. (2019).
[72]
J Scott Long. 1983. Confirmatory factor analysis: A preface to LISREL. Vol. 33. Sage Publications.
[73]
Robert Loo. 1983. Caveat on Sample Sizes in Factor Analysis. Perceptual and Motor Skills 56, 2 (1983), 371--374.
[74]
Robert C MacCallum, Michael W Browne, and Hazuki M Sugawara. 1996. Power analysis and determination of sample size for covariance structure modeling. Psychological methods 1, 2 (1996), 130.
[75]
LA Maruscuilo and JR Levin. 1983. Multivariate statistics in the social sciences. Books/Cole, Monterrey, California (1983).
[76]
Gary McGraw, Sammy Migues, and Jacob West. 2018. Building Security in Maturity Model. Technical Report. Open Web Application Security Project.
[77]
Robert K McKinley, Terjinder Manku-Scott, Adrian M Hastings, David P French, and Richard Baker. 1997. Reliability and validity of a new measure of patient satisfaction with out of hours primary medical care in the united kingdom: development of a patient questionnaire. BMJ 314, 7075 (1997), 193.
[78]
Adam W Meade and S Bartholomew Craig. 2012. Identifying careless responses in survey data. Psychological methods 17, 3 (2012), 437.
[79]
Meetup. 2019. We are what we do | Meetup. https://www.meetup.com/. (2019).
[80]
Microsoft. 2019. Microsoft Security Development Lifecycle Practices. https://www.microsoft.com/en-us/securityengineering/sdl/practices. (2019).
[81]
Alena Naiakshina, Anastasia Danilova, Eva Gerlitz, Emanuel von Zezschwitz, and Matthew Smith. 2019. "If You Want, I Can Store the Encrypted Password": A Password-Storage Field Study with Freelance Developers. In Proceedings of the 37th CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 140, 12 pages.
[82]
Alena Naiakshina, Anastasia Danilova, Christian Tiefenau, Marco Herzog, Sergej Dechand, and Matthew Smith. 2017. Why Do Developers Get Password Storage Wrong?: A Qualitative Usability Study. In Proceedings of the 24th ACM SIGSAC Conference on Computer and Communications Security (CCS '17). ACM, NY, NY, USA, 311--328.
[83]
Alena Naiakshina, Anastasia Danilova, Christian Tiefenau, and Matthew Smith. 2018. Deception Task Design in Developer Password Studies: Exploring a Student Sample. In Proceedings of the 14th Symposium on Usable Privacy and Security (SOUPS '18). USENIX Association, Baltimore, MD, 297--313. https://www.usenix.org/conference/soups2018/presentation/naiakshina
[84]
Richard G Netemeyer, William O Bearden, and Subhash Sharma. 2003. Scaling procedures: Issues and applications. Sage Publications.
[85]
William Newhouse, Stephanie Keith, Benjamin Scribner, and Greg Witte. 2017. NIST Special Publication 800--181, The NICE Cybersecurity Workforce Framework. Technical Report. National Institute of Standards and Technology.
[86]
Jum C Nunnally. 1994. Psychometric theory (3rd ed.). Tata McGraw-Hill Education.
[87]
National Institute of Standards and Technology. 2019. Software and Supply Chain Assurance Forum - Cyber Supply Chain Risk Management | CSRC. https://csrc.nist.gov/Projects/Supply-Chain-Risk-Management/SSCA. (2019).
[88]
Daniela Seabra Oliveira, Tian Lin, Muhammad Sajidur Rahman, Rad Akefirad, Donovan Ellis, Eliany Perez, Rahul Bobhate, Lois A. DeLong, Justin Cappos, and Yuriy Brun. 2018. API Blindspots: Why Experienced Developers Write Vulnerable Code. In Proceedings of the 14th Symposium on Usable Privacy and Security (SOUPS '18). USENIX Association, Baltimore, MD, 315--328. https://www.usenix.org/conference/soups2018/presentation/oliveira
[89]
Eyal Peer, Laura Brandimarte, Sonam Samat, and Alessandro Acquisti. 2017. Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology 70 (2017), 153 -- 163.
[90]
K. Qian, D. Lo, H. Shahriar, L. Li, F. Wu, and P. Bhattacharya. 2017. Learning database security with hands-on mobile labs. In 2017 IEEE Frontiers in Education Conference (FIE). 1--6.
[91]
Adrian E Raftery. 1995. Bayesian model selection in social research. Sociological methodology (1995), 111--163.
[92]
Gilles Raîche, Theodore A. Walls, David Magis, Martin Riopel, and Jean-Guy Blais. 2013. Non-Graphical Solutions for Cattell's Scree Test. Methodology 9, 1 (2013), 23--29.
[93]
Tenko Raykov. 1997. Scale Reliability, Cronbach's Coefficient Alpha, and Violations of Essential Tau-Equivalence with Fixed Congeneric Components. Multivariate Behavioral Research 32, 4 (1997), 329--353. 26777071.
[94]
Tony Rice, Josh Brown-White, Tania Skinner, Nick Ozmore, Nazira Carlage, Wendy Poland, Eric Heitzman, and Danny Dhillon. 2018. Fundamental Practices for Secure Software Development. Technical Report. Software Assurance Forum for Excellence in Code.
[95]
John P Robinson, Phillip R Shaver, and Lawrence S Wrightsman. 1991. Criteria for scale selection and evaluation. Measures of personality and social psychological attitudes 1, 3 (1991), 1--16.
[96]
Dale C. Rowe, Barry M. Lunt, and Joseph J. Ekstrom. 2011. The Role of Cyber-security in Information Technology Education. In Proceedings of the 12th Conference on Information Technology Education (SIGITE '11). ACM, NY, NY, USA, 113--122.
[97]
Andrew Ruef, Michael Hicks, James Parker, Dave Levin, Michelle L. Mazurek, and Piotr Mardziel. 2016. Build It, Break It, Fix It: Contesting Secure Development. In Proceedings of the 23rd ACM SIGSAC Conference on Computer and Communications Security (CCS '16). ACM, NY, NY, USA, 690--703.
[98]
Caitlin Sadowski, Jeffrey van Gogh, Ciera Jaspan, Emma Söderberg, and Collin Winter. 2015. Tricorder: Building a Program Analysis Ecosystem. In Proceedings of the 37th International Conference on Software Engineering (ICSE '15). IEEE Press, 598--608.
[99]
Ralf Schwarzer, Matthias Jerusalem, J Weinman, S Wright, and M Johnston. 1995. Measures in health psychology: A user's portfolio. Causal and control beliefs. Generalized Self-Efficacy Scal, NFER-NELSON, Windsor (1995), 35--37.
[100]
Swapneel Sheth, Gail Kaiser, and Walid Maalej. 2014. Us and Them: A Study of Privacy Requirements Across North America, Asia, and Europe. In Proceedings of the 36th International Conference on Software Engineering (ICSE 2014). ACM, NY, NY, USA, 859--870.
[101]
Yan Shoshitaishvili, Michael Weissbacher, Lukas Dresel, Christopher Salls, Ruoyu Wang, Christopher Kruegel, and Giovanni Vigna. 2017. Rise of the HaCRS: Augmenting Autonomous Cyber Reasoning Systems with Human Assistance. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS '17). ACM, Dallas, Texas, USA, 347--362.
[102]
Justin Smith, Brittany Johnson, Emerson Murphy-Hill, Bill Chu, and Heather Richter Lipford. 2015. Questions Developers Ask While Diagnosing Potential Security Vulnerabilities with Static Analysis. In Proceedings of the 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE '15). ACM, 248--259.
[103]
George W Snedecor and Witiiam G Cochran. 1989. Statistical methods. Ames: Iowa State Univ. Press Iowa (1989).
[104]
StackOverflow. 2018. StackOverflow Developer Survey Results 2018. https://insights.stackoverflow.com/survey/2018. (2018).
[105]
Barbara G Tabachnick, Linda S Fidell, and Jodie B Ullman. 2007. Using multivariate statistics. Vol. 5. Pearson Boston, MA.
[106]
Tyler W. Thomas, Heather Lipford, Bill Chu, Justin Smith, and Emerson Murphy-Hill. 2016. What Questions Remain? An Examination of How Developers Understand an Interactive Static Analysis Tool. In Proceedings of the 2nd Workshop on Security Information Workers (WSIW '16). USENIX Association, Denver, CO. https://www.usenix.org/conference/soups2016/ workshop-program/wsiw16/presentation/thomas
[107]
Meng-Jung Tsai, Ching-Yeh Wang, and Po-Fen Hsu. 2019. Developing the Computer Programming Self-Efficacy Scale for Computer Literacy Education. Journal of Educational Computing Research 56, 8 (2019), 1345--1360.
[108]
Li tze Hu and Peter M. Bentler. 1999. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6, 1 (1999), 1--55.
[109]
Upworki. 2017. Hire Freelancers, Make Things Happen | Upwork. https://upwork.com. (2017).
[110]
Wayne F. Velicer, Andrew C. Peacock, and Douglas N. Jackson. 1982. A Comparison Of Component And Factor Patterns: A Monte Carlo Approach. Multivariate Behavioral Research 17, 3 (1982), 371--388. 26800757.
[111]
Markos Viggiato, Ricardo Terra, Henrique Rocha, Marco Tulio Valente, and Eduardo Figueiredo. 2018. Microservices in Practice: A Survey Study. (2018).
[112]
Daniel Votipka, Rock Stevens, Elissa M. Redmiles, Jeremy Hu, and Michelle L. Mazurek. 2018. Hackers vs. Testers: A Comparison of Software Vulnerability Discovery Processes. In Proceedings of the 39th IEEE Symposium on Security and Privacy (SP '18). San Francisco, CA, 374--391.
[113]
Stacey Watson and Heather Richter Lipford. 2017. A Proposed Visualization for Vulnerability Scan Data. In Proceedings of the 3rd Workshop on Security Information Workers (WSIW '17). USENIX Association, Santa Clara, CA. https://www.usenix.org/conference/soups2017/workshop-program/wsiw2017/watson
[114]
Susan C Weller and A Kimball Romney. 1988. Systematic data collection. Vol. 10. Sage publications.
[115]
Joseph Werther, Michael Zhivich, Tim Leek, and Nickolai Zeldovich. 2011. Experiences in Cyber Security Education: The MIT Lincoln Laboratory Capture-the-flag Exercise. In Proc. of the 4th Conference on Cyber Security Experimentation and Test (CSET'11). USENIX Association, Berkeley, CA, USA, 12--12. http://dl.acm.org/citation.cfm?id=2027999.2028011
[116]
Chamila Wijayarathna and Nalin A. G. Arachchilage. 2018. Why Johnny Can't Store Passwords Securely?: A Usability Evaluation of Bouncycastle Password Hashing. In Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering (EASE'18). ACM, NY, NY, USA, 205--210.
[117]
Jim Witschey, Olga Zielinska, Allaire Welk, Emerson Murphy-Hill, Chris Mayhorn, and Thomas Zimmermann. 2015. Quantifying Developers' Adoption of Security Tools. In Proceedings of the 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE '15). ACM, 260--271.
[118]
Irene M.Y. Woon and Atreyi Kankanhalli. 2007. Investigation of IS professionals' intention to practise secure development of applications. International Journal of Human-Computer Studies 65, 1 (2007), 29 -- 41.

Cited By

View all
  • (2024)A survey of cybersecurity professionals' perceptions and experiences of safety and belonging in the communityProceedings of the Twentieth USENIX Conference on Usable Privacy and Security10.5555/3696899.3696900(1-20)Online publication date: 12-Aug-2024
  • (2024)Towards Security-Focused Developer PersonasProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685406(1-18)Online publication date: 13-Oct-2024
  • (2024)NERDS: A Non-invasive Environment for Remote Developer StudiesProceedings of the 17th Cyber Security Experimentation and Test Workshop10.1145/3675741.3675750(74-82)Online publication date: 13-Aug-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
April 2020
10688 pages
ISBN:9781450367080
DOI:10.1145/3313831
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. scale development
  2. secure development

Qualifiers

  • Research-article

Funding Sources

  • National Science Foundation

Conference

CHI '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)141
  • Downloads (Last 6 weeks)16
Reflects downloads up to 06 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A survey of cybersecurity professionals' perceptions and experiences of safety and belonging in the communityProceedings of the Twentieth USENIX Conference on Usable Privacy and Security10.5555/3696899.3696900(1-20)Online publication date: 12-Aug-2024
  • (2024)Towards Security-Focused Developer PersonasProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685406(1-18)Online publication date: 13-Oct-2024
  • (2024)NERDS: A Non-invasive Environment for Remote Developer StudiesProceedings of the 17th Cyber Security Experimentation and Test Workshop10.1145/3675741.3675750(74-82)Online publication date: 13-Aug-2024
  • (2024)A Qualitative Analysis of Practical De-Identification GuidesProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3690270(1611-1625)Online publication date: 2-Dec-2024
  • (2024)From Prisons to Programming: Fostering Self-Efficacy via Virtual Web Design Curricula in Prisons and JailsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642717(1-13)Online publication date: 11-May-2024
  • (2024)Taking 5 minutes protects you for 5 monthsComputers and Security10.1016/j.cose.2023.103620137:COnline publication date: 1-Feb-2024
  • (2023)"Security is not my field, I'm a stats guy"Proceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620448(3763-3780)Online publication date: 9-Aug-2023
  • (2023)Factors of Haptic Experience across Multiple Haptic ModalitiesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581514(1-12)Online publication date: 19-Apr-2023
  • (2023)A Usability Evaluation of AFL and libFuzzer with CS StudentsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581178(1-18)Online publication date: 19-Apr-2023
  • (2023)Stuck in the Permissions With You: Developer & End-User Perspectives on App Permissions & Their Privacy RamificationsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581060(1-24)Online publication date: 19-Apr-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media