Abstract
The development of Machine Learning (ML) models is more than just a special case of software development (SD): ML models acquire properties and fulfill requirements even without direct human interaction in a seemingly uncontrollable manner. Nonetheless, the underlying processes can be described in a formal way. We define a comprehensive SD process model for ML that encompasses most tasks and artifacts described in the literature in a consistent way. In addition to the production of the necessary artifacts, we also focus on generating and validating fitting descriptions in the form of specifications. We stress the importance of further evolving the ML model throughout its life-cycle even after initial training and testing. Thus, we provide various interaction points with standard SD processes in which ML often is an encapsulated task. Further, our SD process model allows to formulate ML as a (meta-) optimization problem. If automated rigorously, it can be used to realize self-adaptive autonomous systems. Finally, our SD process model features a description of time that allows to reason about the progress within ML development processes. This might lead to further applications of formal methods within the field of ML.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
IEEE standard for system and software verification and validation. IEEE Std. 1012–2012, pp. 1–223 (2012). https://doi.org/10.1109/IEEESTD.2012.6204026
Akkiraju, R., et al.: Characterizing machine learning processes: a maturity framework. In: Fahland, D., Ghidini, C., Becker, J., Dumas, M. (eds.) BPM 2020. LNCS, vol. 12168, pp. 17–31. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58666-9_2
Amershi, S., et al.: Software engineering for machine learning: a case study. In: 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pp. 291–300 (2019)
Bernon, C., Camps, V., Gleizes, M.P., Picard, G.: Engineering self-adaptive multi-agent systems: the adelfe methodology. In: Agent-Oriented Methodologies, vol. 7, pp. 172–202. Idea Group Publishing (2005)
Bosch, J., Crnkovic, I., Olsson, H.H.: Engineering AI systems: a research agenda. arxiv:2001.07522 (2020)
Bourque, P., Fairley, R.E. (eds.): SWEBOK: guide to the software engineering body of knowledge. IEEE Computer Society, version 3.0 edn (2014). https://www.swebok.org
Fainekos, G., Hoxha, B., Sankaranarayanan, S.: Robustness of Specifications and its applications to falsification, parameter mining, and runtime monitoring with S-TaLiRo. In: Finkbeiner, B., Mariani, L. (eds.) RV 2019. LNCS, vol. 11757, pp. 27–47. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32079-9_3
Gabor, T., et al.: The scenario coevolution paradigm: adaptive quality assurance for adaptive systems. Int. J. Softw. Tools Technology Transfer 22(4), 457–476 (2020)
Garlan, D., Schmerl, B., Cheng, S.W.: Software Architecture Based Self Adaptation, pp. 31–55. Springer, Boston (2009). https://doi.org/10.1007/978-0-387-89828-5_2
Geihs, K.: Selbst-adaptive software. Informatik-Spektrum 31(2), 133–145 (2008)
Giray, G.: A software engineering perspective on engineering machine learning systems: state of the art and challenges. J. Syst. Softw. 180, 111031 (2021)
He, X., Zhao, K., Chu, X.: AutoML: a survey of the state-of-the-art. Knowl.-Based Syst. 212, 106622 (2021)
Hernandez, D., Brown, T.B.: Measuring the algorithmic efficiency of neural networks. arxiv:2005.04305 (2020)
Hölzl, M., Koch, N., Puviani, M., Wirsing, M., Zambonelli, F.: The ensemble development life cycle and best practices for collective autonomic systems. In: Wirsing, M., Hölzl, M., Koch, N., Mayer, P. (eds.) Software Engineering for Collective Autonomic Systems. LNCS, vol. 8998, pp. 325–354. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16310-9_9
Kreuzberger, D., Kühl, N., Hirschl, S.: Machine learning operations (mlops): overview, definition, and architecture. arxiv:2205.02302 (2022). https://doi.org/10.48550/ARXIV.2205.02302
Kröger, F., Merz, S.: Temporal Logic and State Systems. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68635-4
Kruchten, P.: The Rational Unified Process-An Introduction (2000)
Linardatos, P., Papastefanopoulos, V., Kotsiantis, S.: Explainable AI: a review of machine learning interpretability methods. Entropy 23(1), 18 (2021)
Lwakatare, L.E., Crnkovic, I., Bosch, J.: DevOps for AI-challenges in development of ai-enabled applications. In: 2020 International Conference on Software, Telecommunications and Computer Networks (SoftCOM), pp. 1–6. IEEE (2020)
Lwakatare, L.E., Raj, A., Bosch, J., Olsson, H., Crnkovic, I.: A taxonomy of software engineering challenges for machine learning systems: an empirical investigation, pp. 227–243 (2019)
Martínez-Fernández, S., et al.: Software engineering for AI-based systems: a survey. arxiv:2105.01984 (2021)
McKinley, P., Sadjadi, S., Kasten, E., Cheng, B.: Composing adaptive software. Computer 37(7), 56–64 (2004)
Müller., R., et al.: Acoustic leak detection in water networks. In: Proceedings of the 13th International Conference on Agents and Artificial Intelligence, vol. 2: ICAART, pp. 306–313 (2021). https://doi.org/10.5220/0010295403060313
Pappagallo, A., Massini, A., Tronci, E.: Monte carlo based statistical model checking of cyber-physical systems: a review. Information 11(12), 588 (2020)
Paullada, A., Raji, I.D., Bender, E.M., Denton, E., Hanna, A.: Data and its (dis)contents: a survey of dataset development and use in machine learning research. Patterns 2(11), 100336 (2021)
Phan, T., et al.: Learning and testing resilience in cooperative multi-agent systems. In: Proceedings of the 19th Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2020 (2020)
Rahman, M.S., Rivera, E., Khomh, F., Guéhéneuc, Y., Lehnert, B.: Machine learning software engineering in practice: an ind. case study. arXiv:1906.07154 (2019)
Ribeiro, M.T., Singh, S., Guestrin, C.: “Why Should I Trust You?”: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD, KDD 2016, pp. 1135–1144. ACM (2016)
Ritz, F., et al.: Specification aware multi-agent reinforcement learning. In: Agents and Artificial Intelligence, pp. 3–21. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-10161-8_1
Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1(5), 206–215 (2019)
Sevilla, J., Villalobos, P.: Parameter counts in machine learning. AI Alignment Forum (2021). https://www.alignmentforum.org/posts/GzoWcYibWYwJva8aL
Sinreich, D.: An architectural blueprint for autonomic computing (2006). https://www-03.ibm.com/autonomic/pdfs/AC%20Blueprint%20White%20Paper%20V7.pdf
Studer, S., et al.: Towards CRISP-ML(Q): a machine learning process model with quality assurance methodology. Mach. Learn. Knowl. Extract. 3(2), 392–413 (2021)
Watanabe, Y., et al.: Preliminary systematic literature review of machine learning system development process. arxiv:1910.05528 (2019)
Wirsing, M., Belzner, L.: Towards systematically engineering autonomous systems using reinforcement learning and planning. In: Proceedings of Analysis, Verification and Transformation for Declarative Programming and Intelligent Systems (AVERTIS) (2022). https://doi.org/10.13140/RG.2.2.10618.16328
Wirsing, M., Hölzl, M., Koch, N., Mayer, P. (eds.): Software Engineering for Collective Autonomic Systems. LNCS, vol. 8998. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16310-9
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ritz, F. et al. (2022). Capturing Dependencies Within Machine Learning via a Formal Process Model. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation. Adaptation and Learning. ISoLA 2022. Lecture Notes in Computer Science, vol 13703. Springer, Cham. https://doi.org/10.1007/978-3-031-19759-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-19759-8_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19758-1
Online ISBN: 978-3-031-19759-8
eBook Packages: Computer ScienceComputer Science (R0)