Abstract
Over the last fifteen years, various frameworks for data-aware process modelling have been proposed, several of which provide a set of evaluation criteria but which differ in their focus, the terminology used, the level of detail used to describe their criteria and how these are evaluated. In addition, there are well-established evaluation frameworks of a more general nature that can be applied to data-centric process modelling too. A comprehensive and unbiased evaluation framework for (multi-)modelling approaches that also caters for more general aspects such as understandability, ease of use, model quality, etc., does not yet exist and is therefore the research gap addressed in this paper. This paper addresses this gap by using existing evaluation frameworks and developing a taxonomy that is used to categorise all the criteria from existing evaluation frameworks. The results are then discussed and related to the challenges and concerns identified by practitioners.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Hull, R.: Artifact-centric business process models: brief survey of research results and challenges. In: Meersman, R., Tari, Z. (eds.) OTM 2008. LNCS, vol. 5332, pp. 1152–1163. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88873-4_17
Künzle, V., Reichert, M.: PHILharmonicFlows: towards a framework for object-aware process management. J. Softw. Maint. Evol. Res. Pract. 23(4), 205–244 (2011). https://doi.org/10.1002/smr.524
Steinau, S., Marrella, A., Andrews, K., Leotta, F., Mecella, M., Reichert, M.: DALEC: a framework for the systematic evaluation of data-centric approaches to process management software. Softw. Syst. Model. 18(4), 2679–2716 (2018). https://doi.org/10.1007/s10270-018-0695-0
Moody, D.L.: The method evaluation model: a theoretical model for validating information systems design methods. In: ECIS 2003 Proceedings, 2003, p. 79. [Online]. https://aisel.aisnet.org/ecis2003/79
Krogstie, J., Sindre, G., Jørgensen, H.: Process models representing knowledge for action: a revised quality framework. Eur. J. Inf. Syst. 15(1), 91–102 (2006). https://doi.org/10.1057/palgrave.ejis.3000598
Reijers, H.A., et al.: Evaluating data-centric process approaches: does the human factor factor in? Softw. Syst. Model. 16(3), 649–662 (2016). https://doi.org/10.1007/s10270-015-0491-z
Davis, F.D.: A technology Acceptance Model For Empirically Testing New End-User Information Systems: Theory and Results. Massachusetts Institute of Technology (1985)
Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 18, 319–340 (1989)
Venkatesh, V., Thong, J.Y.L., Xu, X.: Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Q. 36(1), 157–178 (2012). https://doi.org/10.2307/41410412
Lindland, O.I., Sindre, G., Solvberg, A.: Understanding quality in conceptual modeling. IEEE Softw 11(2), 42–49 (1994). https://doi.org/10.1109/52.268955
Krogstie, J., Lindland, O.I., Sindre, G.: Defining quality aspects for conceptual models. In: Falkenberg, E.D., Hesse, W., Olivé, A. (eds.) Information System Concepts. IAICT, pp. 216–231. Springer, Boston, MA (1995). https://doi.org/10.1007/978-0-387-34870-4_22
Krogstie, J., Jørgensen, H.D.: Quality of interactive models. In: Olivé, A., Yoshikawa, M., Yu, E.S.K. (eds.), Advanced Conceptual Modeling Techniques, Berlin, Heidelberg: Springer Berlin Heidelberg, 2003, pp. 351–363 (2003). https://doi.org/10.1007/978-3-540-45275-1_31
Nelson, H.J., Poels, G., Genero, M., Piattini, M.: A conceptual modeling quality framework. Softw. Qual. J. 20(1), 201–228 (2012). https://doi.org/10.1007/s11219-011-9136-9
Wand, Y., Weber, R.: An ontological model of an information system. IEEE Trans. Softw. Eng. 16(11), 1282–1292 (1990). https://doi.org/10.1109/32.60316
Künzle, V., Weber, B., Reichert, M.: Object-aware business processes: properties, requirements, existing approaches. University of Ulm (2010)
Giraldo, F.D., España, S., Giraldo, W.J., Pastor, Ó.: Evaluating the quality of a set of modelling languages used in combination: a method and a tool. Inf Syst 77, 48–70 (2018). https://doi.org/10.1016/J.IS.2018.06.002
Nickerson, R.C., Varshney, U., Muntermann, J.: A method for taxonomy development and its application in information systems. Eur. J. Inf. Syst. 22(3), 336–359 (2013)
van der Aalst, W.M.P.: Business process management: a comprehensive survey. ISRN Softw. Eng. 2013, 507984 (2013). https://doi.org/10.1155/2013/507984
ISO: ISO/IEC/IEEE 42010:2011 Systems and software engineering — Architecture description. [Online]. https://www.iso.org/standard/50508.html. Accessed 9 Mar 2023
Bernaert, M., Poels, G., Snoeck, M., De Backer, M.: CHOOSE: towards a metamodel for enterprise architecture in small and medium-sized enterprises. Inf. Syst. Front. 18(4), 781–818 (2015). https://doi.org/10.1007/s10796-015-9559-0
Kleppe, A.G.: A language description is more than a Metamodel. In: 4th International Workshop on Software Language Engineering, ATEM 2007 (2007)
Ruiz, J., Serral, E., Snoeck, M.: Evaluating user interface generation approaches: model-based versus model-driven development. Softw. Syst. Model. 18(4), 2753–2776 (2018). https://doi.org/10.1007/s10270-018-0698-x
Verbruggen, C., Snoeck, M.: Practitioners’ experiences with model-driven engineering: a meta-review. Softw. Syst. Model 22(1), 111–129 (2023). https://doi.org/10.1007/s10270-022-01020-1
Moody, D.: The physics of notations: toward a scientific basis for constructing visual notations in software engineering. IEEE Trans. Softw. Eng. 35(6), (2009). https://doi.org/10.1109/TSE.2009.67
Börstler, J., bin Ali, N., Svensson, M., Petersen, K.: Investigating acceptance behavior in software engineering—theoretical perspectives. J. Syst. Softw. 198, 111592 (2023)
Acknowledgement
This research has been funded by the KU Leuven research fund, grant C17421017.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Verbruggen, C., Snoeck, M. (2023). TEC-MAP: A Taxonomy of Evaluation Criteria for Multi-modelling Approaches. In: van der Aa, H., Bork, D., Proper, H.A., Schmidt, R. (eds) Enterprise, Business-Process and Information Systems Modeling. BPMDS EMMSAD 2023 2023. Lecture Notes in Business Information Processing, vol 479. Springer, Cham. https://doi.org/10.1007/978-3-031-34241-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-34241-7_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34240-0
Online ISBN: 978-3-031-34241-7
eBook Packages: Computer ScienceComputer Science (R0)