More Web Proxy on the site http://driver.im/

short-paper

Do Models Improve the Understanding of Safety Compliance Needs?: Insights from a Pilot Experiment

Authors:

Jose Luis de la Vara,

Beatriz Marín,

Giovanni Giachetti,

Clara AyoraAuthors Info & Claims

ESEM '16: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Article No.: 32, Pages 1 - 6

https://doi.org/10.1145/2961111.2962621

Published: 08 September 2016 Publication History

Abstract

Context. Many critical systems must meet safety compliance needs from safety standards. These standards are usually large textual documents whose compliance needs can be hard to understand. As a solution, the use of models has been proposed. Goal. We aim to provide evidence of the extent to which models improve the understanding of safety compliance needs. Method. We designed an experiment and ran a pilot to study the effectiveness, efficiency, and perceived benefits of understanding these needs, with the text of standards and with models in the form of UML object diagrams. Results. The overall results from 15 Bachelor students show that the effectiveness of understanding safety compliance needs increases very little with models (2%), and the efficiency even decreases (24%). Nonetheless, the results improve when the potential complexity in navigating the models is taken into account (15% effectiveness increase). The students find benefits in using the models but most consider that the models are hard to understand. Conclusions. The extent to which models improve the understanding of safety compliance needs seems to be lower than what the research community expects. New studies are necessary to confirm our initial insights.

References

[1]

Abdulkhaleq, A. and Wagner, S. 2015. A controlled experiment for the empirical evaluation of safety analysis techniques for safety-critical software. In EASE 2015.

Digital Library

[2]

Abrahão, S. et al. 2013. Assessing the Effectiveness of Sequence Diagrams in the Comprehension of Functional Requirements. IEEE T. Software Eng. 39, 3, 327--342.

Digital Library

[3]

Briand, L. et al. 2014. Traceability and SysML design slices to support safety inspections: A controlled experiment. ACM T. Softw. Eng. Meth. 23, 1, 9:1-9:43.

Digital Library

[4]

Cruz-Lemus, J. A. et al. 2009. Assessing the understandability of UML statechart diagrams with composite states. Empir. Softw. Eng. 14, 6, 685--719.

Digital Library

[5]

Cyra, L. and Górski, J. 2008. Expert Assessment of Arguments. In SAFECOMP 2008.

[6]

de la Vara, J. L. et al. 2016. An Industrial Survey on Safety Evidence Change Impact Analysis Practice. IEEE T. Software Eng. (accepted paper; preprint available)

[7]

de la Vara, J. L. et al. 2016. Model-based specification of safety compliance needs for critical systems: A holistic generic metamodel. Inform. Software Tech. 72, 16--30.

Digital Library

[8]

De Lucia, A. et al. 2010. An experimental comparison of ER and UML class diagrams for data modelling. Empir. Softw. Eng.15, 5, 455--492.

Digital Library

[9]

ESA. 2006. Software engineering and standardisation. http://www.esa.int/TEC/Software_engineering_and_standardisation/TECBUCUXBQE_0.html

[10]

Hadar, I. et al. 2013. Comparing the comprehensibility of requirements models expressed in Use Case and Tropos. Inform. Software Tech. 55, 10, 1823--1843.

Digital Library

[11]

Heijstek, W. et al. 2011. Experimental Analysis of Textual and Graphical Representations for Software Architecture Design. In ESEM 2011.

Digital Library

[12]

Jung, J. et al. 2013. Experimental Comparison of Two Safety Analysis Methods and Its Replication. In ESEM 2013.

[13]

Mendling, J. et al. 2012. Factors of process model comprehension. Decis. Support Sys. 53, 1, 195--206.

Digital Library

[14]

Mouaffo, A. et al. 2014. Controlled experiments comparing fault-tree-based safety analysis techniques. In EASE 2014.

Digital Library

[15]

Nair, S. et al. 2014. An extended systematic literature review on provision of evidence for safety certification. Inform. Software Tech. 56, 7, 689--717.

[16]

Nair, S. et al. 2014. Safety Evidence Traceability: Problem Analysis and Model. In REFSQ 2014.

Digital Library

[17]

Nair, S. et al. 2015. Evidence management for compliance of critical systems with safety standards: A survey on the state of practice. Inform. Software Tech. 60, 1--15.

[18]

OMG. 2015. Structured Assurance Case Metamodel (SACM). http://www.omg.org/spec/SACM/

[19]

Ottensooser, A. et al. 2012. Making sense of business process descriptions. J. Syst. Softw. 85, 3, 596--606.

Digital Library

[20]

Panesar-Walawege, R. K. et al. 2013. Supporting the verification of compliance to safety standards via model-driven engineering. Inform. Software Tech.55, 5, 836--864.

Digital Library

[21]

Scanniello, G. et al. 2014. On the effect of using SysML requirement diagrams to comprehend requirements: results from two controlled experiments. In EASE 2014.

Digital Library

[22]

Sharafi, Z. et al. 2013. An empirical study on the efficiency of graphical vs. textual representations in requirements comprehension. In ICPC 2013.

[23]

Stallbaum, H. and Rzepka M. 2010. Toward DO-178B-compliant Test Models. In MoDeVVa 2010.

Digital Library

[24]

Wohlin, C. et al. 2012. Experimentation in Software Engineering. 2nd ed. Springer, Heidelberg.

Digital Library

Cited By

Adedjouma MYakymets N(2019)A Framework for Model-Based Dependability Analysis of Cyber-Physical Systems2019 IEEE 19th International Symposium on High Assurance Systems Engineering (HASE)10.1109/HASE.2019.00022(82-89)Online publication date: Jan-2019
https://doi.org/10.1109/HASE.2019.00022
Labunets KOivo MMéndez DMockus A(2018)No search allowedProceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3239235.3239247(1-10)Online publication date: 11-Oct-2018
https://dl.acm.org/doi/10.1145/3239235.3239247
Labunets KMassaci FTedeschi ABener ATurhan BBiffl S(2017)Graphical vs. tabular notations for risk modelsProceedings of the 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1109/ESEM.2017.40(267-276)Online publication date: 9-Nov-2017
https://dl.acm.org/doi/10.1109/ESEM.2017.40
Show More Cited By

Recommendations

Model-based specification of safety compliance needs for critical systems

We present a metamodel for the specification of safety compliance needs.The metamodel abstracts concepts and criteria from different safety standards.Its application results in models for structuring and managing compliance information.It can be used ...
Assessment of the quality of the text of safety standards with industrial semantic technologies
Highlights
- We present an approach to assess the quality of the text of safety standards.
- The approach is based on text quality analysis with RQA - Quality Studio.
- The approach can detect issues such as imprecision, ambiguity, and ...
Abstract
Most safety-critical systems are subject to rigorous assurance processes to justify that the systems are dependable. These processes are typically conducted in compliance with safety standards, e.g., DO-178C for software in aerospace. This can be ...
Safety Evidence Traceability: Problem Analysis and Model
REFSQ 2014: Proceedings of the 20th International Working Conference on Requirements Engineering: Foundation for Software Quality - Volume 8396

[Context and motivation] Safety evidence plays an important role in gaining confidence in the safe operation of a system in a given context. For a large system, it is necessary to provide information about thousands of artefacts that might be used as ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEM '16: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

September 2016

457 pages

ISBN:9781450344272

DOI:10.1145/2961111

General Chair:
Marcela Genero
University of Castilla-La Mancha, Spain
,
Program Chairs:
Andreas Jedlitschka
Fraunhofer IESE, Germany
,
Magne Jørgensen
Simula Research Laboratory, Norway
,
Giuseppe Scanniello
University of Basilicata, Italy
,
Sreedevi Sampath
University of Maryland Baltimore County, USA
,
Danilo Caivano
SER&Practices, Italy
,
Daniel Port
University of Hawaii, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Funding Sources

Conference

ESEM '16

Sponsor:

SIGSOFT

ESEM '16: ACM/IEEE 9th International Symposium on Empirical Software Engineering and Measurement

September 8 - 9, 2016

Ciudad Real, Spain

Acceptance Rates

ESEM '16 Paper Acceptance Rate 27 of 122 submissions, 22%;

Overall Acceptance Rate 130 of 594 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
181
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Adedjouma MYakymets N(2019)A Framework for Model-Based Dependability Analysis of Cyber-Physical Systems2019 IEEE 19th International Symposium on High Assurance Systems Engineering (HASE)10.1109/HASE.2019.00022(82-89)Online publication date: Jan-2019
https://doi.org/10.1109/HASE.2019.00022
Labunets KOivo MMéndez DMockus A(2018)No search allowedProceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3239235.3239247(1-10)Online publication date: 11-Oct-2018
https://dl.acm.org/doi/10.1145/3239235.3239247
Labunets KMassaci FTedeschi ABener ATurhan BBiffl S(2017)Graphical vs. tabular notations for risk modelsProceedings of the 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1109/ESEM.2017.40(267-276)Online publication date: 9-Nov-2017
https://dl.acm.org/doi/10.1109/ESEM.2017.40
de la Vara JGénova GÁlvarez-Rodríguez JLlorens J(2017)An analysis of safety evidence management with the Structured Assurance Case MetamodelComputer Standards & Interfaces10.1016/j.csi.2016.10.00250:C(179-198)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1016/j.csi.2016.10.002
de la Vara JMarín BAyora CGiachetti G(2017)An Experimental Evaluation of the Understanding of Safety Compliance Needs with ModelsConceptual Modeling10.1007/978-3-319-69904-2_20(239-247)Online publication date: 21-Oct-2017
https://doi.org/10.1007/978-3-319-69904-2_20

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents