Abstract
Identifying security requirements early on can lay the foundation for secure software development. Security requirements are often implied by existing functional requirements but are mostly left unspecified. The Security Discoverer (SD) process automatically identifies security implications of individual requirements sentences and suggests applicable security requirements templates. The objective of this research is to support requirements analysts in identifying security requirements by automating the suggestion of security requirements templates that are implied by existing functional requirements. We conducted a controlled experiment in a graduate-level security class at North Carolina State University (NCSU) to evaluate the SD process in eliciting implied security requirements in 2014. We have subsequently conducted three differentiated replications to evaluate the generalizability and applicability of the initial findings. The replications were conducted across three countries at the University of Trento, NCSU, and the University of Costa Rica. We evaluated the responses of the 205 total participants in terms of quality, coverage, relevance and efficiency. We also develop shared insights regarding the impact of context factors such as time, motivation and support, on the study outcomes and provide lessons learned in conducting the replications. Treatment group, using the SD process, performed significantly better than the control group (at p-value <0.05) in terms of the coverage of the identified security requirements and efficiency of the requirements elicitation process in two of the three replications, supporting the findings of the original study. Participants in the treatment group identified 84 % more security requirements in the oracle as compared to the control group on average. Overall, 80 % of the 111 participants in the treatment group were favorable towards the use of templates in identifying security requirements. Our qualitative findings indicate that participants may be able to differentiate between relevant and extraneous templates suggestions and be more inclined to fill in the templates with additional support. Security requirements templates capture the security knowledge of multiple experts and can support the security requirements elicitation process when automatically suggested, making the implied security requirements more evident. However, individual participants may still miss out on identifying a number of security requirements due to empirical constraints as well as potential limitations on knowledge and security expertise.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
References
Alexander I (2003) Misuse Cases: Use Cases with Hostile Intent. IEEE Softw 20(1):58–66
Braz F, Fernandez EB, VanHilst M (2008) Eliciting security requirements through misuse activities. 4th International Conference on Trust, Privacy & Security in Digital Busines (TrustBus’08), Turin, Italy, September 1–5, 2008, pp 328–333
Carver J (2010) Towards reporting guidelines for experimental replications: a proposal. 1st International Workshop on Replication in Empirical Software Engineering Research (RESER) [Held during ICSE 2010], Cape Town, South Africa
Carver J, Jaccheri L, Morasca S (2010) A checklist for integrating student empirical studies with research and teaching goals. Empir Softw Eng 15:35–59
Carver J, Juristo N, Baldassarre M, Vegas S (2014) Replications of software engineering experiments. Empir Softw Eng 19(2):267–276
De Gramatica M, Labunets K, Massacci F, Paci F, Tedeschi A (2015) The role of catalogues of threats and security controls in security risk assessment: an empirical study with ATM professionals. 21st International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ2015), Springer Verlag, pp 98–114
Fabian B, Gürses S, Heisel M, Santen T, Schmidt H (2010) A comparison of security requirements engineering methods. Requirements Engineering - Special Issue on RE’09: Security Requirements Engineering 15: 7–40
Firesmith DG (2004) Specifying Reusable Security Requirements. J Object Technol 3(1):15
Gray PH, Meister DB (2004) Knowledge sourcing effectiveness. Manag Sci 50(6):821–834
Haley CB, Laney R, Moffett JD, Nuseibeh B (2008) Security requirements engineering: a framework for representation and analysis. IEEE Trans Softw Eng 34(1):133–153
Ito Y, Washizaki H, Yoshizawa M, Fukazawa Y, Okubo T, Kaiya H, Hazeyama A, Yoshioka N, Fernandez E (2015) Systematic mapping of security patterns research. Plop 2015
Karpati P, Opdahl AL, Sindre G (2015) Investigating security threats in architectural context: experimental evaluations of misuse case maps. J Syst Softw 104:90–111. doi:10.1016/j.jss.2015.02.040. Elsevier Ltd
Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE-2007-01 School of Computer Science and Mathematics, Keele University
Lindsay RM, Ehrenberg ASC (1993) The design of replicated studies. Am Stat 47(3):217–228
McCrum-Gardner E (2008) Which is the correct statistical test to use? Br J Oral Maxillofac Surg 46(1):38–41. doi:10.1016/j.bjoms.2007.09.002
McDermott J, Fox C (1999) Using abuse case models for security requirements analysis. In: Computer Security Applications Conference, pp 55–64
Mead NR, Houg ED, Stehney TR (2005) Security quality requirements engineering (SQUARE) methodology. Technical Report CMU/SEI-2005-TR-009 Software Engineering Institute, Carnegie Mellon University
Mellado D, Fernández-Medina E, Piattini M (2007) A common criteria based security requirements engineering process for the development of secure information systems. Comput Stand Interfaces 29(2):244–253
Mellado D, Blanco C, Sánchez LE, Fernández-Medina E (2010) A systematic review of security requirements engineering. Comput Stand Interfaces 32:153–165
Meneely A, Smith B, Williams L (2012) Appendix B: iTrust electronic health care system case study. Software and Systems Traceability, Springer Verlag, pp 425–438
Menzies T, Dekhtyar A, Distefano J, Greenwald J (2007) Problems with precision: a response to “Comments on ‘data mining static code attributes to learn defect predictors”. IEEE Trans Softw Eng 33(9):637–640
Riaz M, King J, Slankas J, Williams L (2014) Hidden in plain sight: automatically identifying security requirements from natural language artifacts. Requirements Engineering (RE 2014). Karlskrona, Sweden, pp 183–192
Riaz M, Slankas J, King J, Williams L (2014) Using templates to elicit implied security requirements from functional requirements − a controlled experiment. International Symposium on Empirical Software Engineering and Measurement (ESEM), Torino, Italy
Riaz M, Breaux T, Williams L (2015) How have we evaluated software pattern application? a systematic mapping study of research design practices. Inf Softw Technol 65:14–38
Riaz M, Elder S, Williams L (2016) Systematically developing prevention, detection, and response patterns for security requirements. 3rd International Workshop on Evolving Security and Privacy Requirements Engineering (ESPRE), Beijing, China
Schumacher M, Fernandez-Buglioni E, Hybertson D, Buschmann F, Sommerlad P (2006) Security Patterns: Integrating Security and Systems Engineering. John Wiley & Sons, Ltd., West Sussex
Sindre G, Opdahl AL (2005) Eliciting security requirements with misuse cases. Requir Eng 10(1):34–44. doi:10.1007/s00766-004-0194-4
Suleiman H, Svetinovic D (2013) Evaluating the effectiveness of the security quality requirements engineering (SQUARE) method: a case study using smart grid advanced metering infrastructure. Requirements Engineering 18(3):251–279
Taubenberger S, Jürjens J, Yu Y, Nuseibeh B (2011) Problem analysis of it-security risk assessment methods – an experience report from the insurance and auditing domain. Future Challenges in Security and Privacy for Academia and Industry, pp 259–270
Taubenberger S, Jürjens J, Yu Y, Nuseibeh B (2013) Resolving vulnerability identification errors using security requirements on business process models. Inf Manag Comput Secur 21(3):202–223
Toval A, Nicolás J, Moros B, García F (2002) Requirements reuse for improving information systems security: a practitioner’s approach. Requir Eng 6(4):205–219
Viera AJ, Garrett JM (2005) Understanding interobserver agreement: the kappa statistic. Fam Med 37(5):360–363
Walia GS, Carver JC (2009) A systematic literature review to identify and classify software requirement errors. Inf Softw Technol 51(7):1087–1109
Wen Y, Zhao H, Liu L (2011) Analysing security requirements patterns based on problems decomposition and composition. First International Workshop on Requirements Patterns (RePa), pp 11–20
Withall S (2007) Software requirement patterns. Microsoft Press
Wohlin C, Runeson P, Höst M, Ohlsson M, Regnell B, Wesslén A (2000) Planning. In: Basili VR (ed) Experimentation in software engineering: an introduction. Kluwer Academic Publishers, Norwell, MA
Yoshioka N, Washizaki H, Maruyama K (2008) A survey on security patterns. Progress in Informatics, Special Issue: The future of software engineering for security and privacy (5): 35–47
Yskout K, Scandariato R, Joosen W (2015) Do security patterns really help designers? Proc. of ICSE 2015. IEEE, pp 292–302
Zhang C, Budgen D (2012) What do we know about the effectiveness of software design patterns? IEEE Trans Softw Eng 38(5):1213–1231
Acknowledgments
This work is partially supported by NSA Science of Security lablet. Fabio Massacci is partially supported by the SESAR Joint Undertaking WP-E EMFASE Project. Christian Quesada-López and Marcelo Jenkins are supported by University of Costa Rica Project No. 834-B5-A18, and Ministry of Science, Technology and Telecommunications (MICITT). Special thanks to Patrick Francis and Patrick Morrison with their help in developing the study oracle. We are thankful to the Realsearch group for their collaboration and helpful comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Andreas Zeller
Rights and permissions
About this article
Cite this article
Riaz, M., King, J., Slankas, J. et al. Identifying the implied: Findings from three differentiated replications on the use of security requirements templates. Empir Software Eng 22, 2127–2178 (2017). https://doi.org/10.1007/s10664-016-9481-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-016-9481-1