Towards the Estimation of Quality Attributes on System Model Histories
Pages 1035 - 1040
Abstract
Companies increasingly rely on Model-Based Systems Engineering to develop Cyber-Physical Systems such as cars, aircraft, or medical devices. The quality of engineering model artifacts is key to efficient collaboration in systems engineering with multi-tier supply chains. Ensuring model artifact quality and comprehensibility for practitioners is challenging. Manual reviews are time- and cost-intensive and subject to bias, whereas existing automated methods based on syntactical rules and model metrics are limited in scope. The paper presents work towards swift quality feedback to system engineers during modeling. The concept allows domain and project-specific context and is applicable to industry-size model artifacts. We implement a data-driven estimation that combines automated model metric extraction with expert quality assessments. We leverage the system model version history from an open-source miniature automotive demonstrator. We assess the model versions' comprehensibility and showcase a semi-automated pipeline to initiate a model quality estimator. We achieve an average accuracy of 0.94 with a random forest approach on our test data.
References
[1]
ISO/IEC JTC 1/SC 7. 2024. Innovation management - Innovation management system - Guidance. https://www.iso.org/standard/78175.html
[2]
Vincent Aravantinos, Sebastian Voss, Sabine Teufl, Florian Hölzl, and Bernhard Schätz. 2015. AutoFOCUS 3: Tooling Concepts for Seamless, Model-based Development of Embedded Systems. In Proc. 8th Int. Workshop Model-based Architecting of Cyber-Physical and Embedded Systems (ACES-MB). 19--26.
[3]
Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman. 2016. Managing Technical Debt in Software Engineering. Dagstuhl Reports 6, 4 (2016), 110--138.
[4]
Francesco Basciani, Juri Di Rocco, Davide Di Ruscio, Ludovico Iovino, and Alfonso Pierantonio. 2019. A tool-supported approach for assessing the quality of modeling artifacts. Journal of Computer Languages 51 (2019), 173--192.
[5]
Konstantin Rupert Blaschke. 2024. Automated Model Quality Estimation and Change Impact Analysis on Model Histories. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings (Lisbon, Portugal) (ICSE-Companion '24). Association for Computing Machinery, New York, NY, USA, 153--155.
[6]
F.J. Domínguez-Mayo, M.J. Escalona, M. Mejías, M. Ross, and G. Staples. 2012. Quality evaluation for Model-Driven Web Engineering methodologies. Information and Software Technology 54, 11 (2012), 1265--1282.
[7]
Marcela G. dos Santos, Bianca M. Napoleão, Fabio Petrillo, Darine Ameyed, and Fehmi Jaafar. 2020. A Preliminary Systematic Mapping on Software Engineering for Robotic Systems: A Software Quality Perspective. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops (IC-SEW'20). Association for Computing Machinery, New York, NY, USA, 647--654.
[8]
John Fitzgerald, Carl Gamble, Peter Gorm Larsen, Kenneth Pierce, and Jim Woodcock. 2015. Cyber-Physical Systems Design: Formal Foundations, Methods and Integrated Tool Chains. In 2015 IEEE/ACM 3rd FME Workshop on Formal Methods in Software Engineering. IEEE, Florence, Italy, 40--46.
[9]
fortiss. 2024. AutoFOCUS3. https://af3.fortiss.org/. Accessed: 2024-07-15.
[10]
fortiss. 2024. Mobility Lab. https://www.fortiss.org/en/research/fortiss-labs/detail/mobility-lab/. Accessed: 2024-07-15.
[11]
Marcela Genero, Mario G. Piattini, and Coral Calero. 2005. A Survey of Metrics for UML Class Diagrams. J. Object Technol. 4 (2005), 59--92.
[12]
Fáber D. Giraldo, Ángela J. Chicaiza, Sergio España, and Óscar Pastor. 2021. Empirical validation of a quality framework for evaluating modelling languages in MDE environments. Software Quality Journal 29, 2 (2021), 275 -- 307.
[13]
Fáber D. Giraldo, Sergio España, and Oscar Pastor. 2014. Analysing the concept of quality in model-driven engineering literature: A systematic review. In 2014 IEEE Eighth International Conference on Research Challenges in Information Science (RCIS). IEEE, Marrakech, Morocco.
[14]
Carlos A. González and Jordi Cabot. 2014. Formal verification of static software models in MDE: A systematic review. Information and Software Technology 56, 8 (2014), 821--838.
[15]
Object Management Group. 2014. Object Constraint Language Specification Version 2.4. https://www.omg.org/spec/OCL/2.4/About-OCL.
[16]
Myron Hecht and Jaron Chen. 2021. Verification and Validation of SysML Models. INCOSE International Symposium 31, 1 (2021), 599--613.
[17]
Paul W. Holland and Samuel Leinhardt. 1971. Transitivity in Structural Models of Small Groups. Comparative Group Studies 2, 2 (1971), 107--124.
[18]
INCOSE. 2015. INCOSE Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities (4th ed.). Wiley-Blackwell, San Diego, CA, USA.
[19]
IncQuery Group. 2024. IncQuery Validator for Enterprise Architect. https://incquery.io/validator-for-enterprise-architect. Accessed: 2024-07-15.
[20]
ISO/IEC/IEEE. 2022. Systems and Software Engineering - Architecture Description. Standard No. ISO/IEC/IEEE 42010:2022. https://www.iso.org/standard/74393.html
[21]
M. G. Kendall. 1938. A New Measure of Rank Correlation. Biometrika 30, 1/2 (1938), 81--93. http://www.jstor.org/stable/2332226
[22]
John Krogstie. 2012. Quality of Models. Springer London, London, 205--247.
[23]
Géza Kulcsár, Johannes Kristan, and Sven Erik Jeroschewski. 2022. Correctness, Completeness, Consistency: Challenges of Migrating Arrowhead System-of-Systems Models. In NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium. IEEE Press, Budapest, Hungary, 1--6.
[24]
Christian F. J. Lange. 2007. Model Size Matters. In Models in Software Engineering, Thomas Kühne (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 211--216.
[25]
Yu Liu, Dong Wang, Jinhua Mi, and He Li (Eds.). 2023. Advances in Reliability and Maintainability Methods and Engineering Applications (1 ed.). Springer Cham, Cham, Switzerland. XXXVII, 622 pages.
[26]
Zipeng Liu, Takayuki Itoh, Jessica Q. Dawson, and Tamara Munzner. 2020. The Sprawlter Graph Readability Metric: Combining Sprawl and Area-Aware Clutter. IEEE Transactions on Visualization and Computer Graphics 26, 6 (June 2020), 2180--2191.
[27]
Törngren M. and Sellgren U. 2018. Complexity Challenges in Development of Cyber-Physical Systems. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10760 LNCS (2018), 478 -- 503.
[28]
Mohamed M.A., Challenger M., and Kardas G. 2020. Applications of model-driven engineering in cyber-physical systems: A systematic mapping study. Journal of Computer Languages 59 (2020).
[29]
Parastoo Mohagheghi, Vegard Dehlen, and Tor Neple. 2009. Definitions and approaches to model quality in model-based software development - A review of literature. Information and Software Technology 51, 12 (2009), 1646--1669. Quality of UML Models.
[30]
Mustafa Abshir Mohamed, Geylani Kardas, and Moharram Challenger. 2021. Model-Driven Engineering Tools and Languages for Cyber-Physical Systems-A Systematic Literature Review. IEEE Access 9 (2021), 48605--48630.
[31]
Gavin J. Mooney, Helen C. Purchase, Michael Wybrow, and Stephen G. Kobourov. 2024. The Multi-Dimensional Landscape of Graph Drawing Metrics. In 2024 IEEE 17th Pacific Visualization Conference (PacificVis). 122--131.
[32]
Mert Ozkaya. 2019. Are the UML modelling tools powerful enough for practitioners? A literature review. IET Software 13, 5 (2019), 338--354.
[33]
Karl Pearson. 1896. VII. Mathematical contributions to the theory of evolution.---III. Regression, heredity, and panmixia. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character 187 (1896), 253--318.
[34]
Klaus Pohl, Harald Hönninger, Reinhold Achatz, and Manfred Broy. 2012. ModelBased Engineering of Embedded Systems (1 ed.). Springer Berlin, Heidelberg, Berlin, Germany. XII, 304 pages. Published: 09 November 2012 (Hardcover), 08 November 2012 (eBook), 14 December 2014 (Softcover).
[35]
Saheed Popoola. 2021. Change Analysis Across Version Histories of Systems Models. Ph. D. Dissertation. The University of Alabama. http://ir.ua.edu/handle/123456789/8198
[36]
D.K.K Shyamal, P.P.G.D Asanka, and D. Wickramaarachchi. 2023. A Comprehensive Approach to Evaluating Software Code Quality Through a Flexible Quality Model. In 2023 International Research Conference on Smart Computing and Systems Engineering (SCSE), Vol. 6. IEEE, Kelaniya, Sri Lanka, 1--8.
[37]
C. Spearman. 1904. The Proof and Measurement of Association between Two Things. The American Journal of Psychology 15, 1 (1904), 72--101. http://www.jstor.org/stable/1412159
[38]
Martin Vogel, Peter Knapik, Moritz Cohrs, Bernd Szyperrek, Winfried Pueschel, Haiko Etzel, Daniel Fiebig, Andreas Rausch, and Marco Kuhrmann. 2021. Metrics in automotive software development: A systematic literature review. Journal of Software: Evolution and Process 33, 2 (2021), e2296. e2296 smr.2296.
[39]
Danny Weyns, Tomas Bures, Radu Calinescu, Barnaby Craggs, John Fitzgerald, David Garlan, Bashar Nuseibeh, Liliana Pasquale, Awais Rashid, Ivan Ruchkin, and Bradley Schmerl. 2021. Six Software Engineering Principles for Smarter Cyber-Physical Systems. In 2021 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C). IEEE, DC, USA, 198--203.
[40]
Xi Zheng and Christine Julien. 2015. Verification and Validation in Cyber Physical Systems: Research Challenges and a Way Forward. In 2015 IEEE/ACM 1st International Workshop on Software Engineering for Smart Cyber-Physical Systems. IEEE, Florence, Italy, 15--18.
Index Terms
- Towards the Estimation of Quality Attributes on System Model Histories
Recommendations
Automated Model Quality Estimation and Change Impact Analysis on Model Histories
ICSE-Companion '24: Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion ProceedingsCyber-Physical Systems integrate hardware with software in complex applications. To mitigate the complexity, engineers rely on model-based systems engineering approaches. Updates and function enhancements lead to frequently changing design constraints ...
A tool environment for quality assurance based on the Eclipse Modeling Framework
The paradigm of model-based software development has become more and more popular since it promises an increase in the efficiency and quality of software development. Following this paradigm, models become primary artifacts in the software development ...
Operationalised product quality models and assessment
ContextSoftware quality models provide either abstract quality characteristics or concrete quality measurements; there is no seamless integration of these two aspects. Quality assessment approaches are, hence, also very specific or remain abstract. ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2024
1261 pages
ISBN:9798400706226
DOI:10.1145/3652620
- Chairs:
- Manuel Wimmer,
- Alexander Egyed,
- Program Chairs:
- Benoit Combemale,
- Marsha Chechik
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
- SIGSOFT: ACM Special Interest Group on Software Engineering
- Johannes Kepler University Linz
- IEEE CS
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 31 October 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
Funding Sources
- BMWK
- BMBF
Conference
MODELS Companion '24
Sponsor:
MODELS Companion '24: ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems
September 22 - 27, 2024
Linz, Austria
Acceptance Rates
Overall Acceptance Rate 144 of 506 submissions, 28%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 34Total Downloads
- Downloads (Last 12 months)34
- Downloads (Last 6 weeks)27
Reflects downloads up to 14 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in