Abstract
Given their vital roles in model-based software engineering, the performance of model-related operations (MOs, such as model transformations) must be systematically tested. However, how to produce a set of large input models that conform to structure-related constraints presents a major challenge to such test. This paper proposes a template-based approach to efficient model generation. Firstly, a DSL is provided to describe templates that specify how to generate a valid model that conforms to structure-related constraints. Secondly, a folding semantic is defined to convert templates into a wrapper metamodel. Thirdly, a wrapper model is generated using the existing model generators (e.g., a random model generator) according to the wrapper metamodel. Fourthly, an unfolding semantics is specified to translate the wrapper model into the desired test input. This paper also presents five case studies to evaluate the proposed approach, and the results demonstrate that such approach can generate large models based on structure-related constraints and facilitate the performance testing of MOs.
Similar content being viewed by others
Notes
The source files of these case studies are downloadable at https://bitbucket.org/ustbmde/model-generation/wiki/SoSyM-Examples.
References
Ali, S., Iqbal, M.Z.Z., Arcuri, A., Briand, L.C.: Generating test data from OCL constraints with search techniques. IEEE Trans. Software Eng. 39(10), 1376–1402 (2013). https://doi.org/10.1109/TSE.2013.17
Anastasakis, K., Bordbar, B., Georg, G., Ray, I.: On challenges of model transformation from UML to alloy. Softw. Syst. Model. 9(1), 69–86 (2010)
Arendt, T., Biermann, E., Jurack, S., Krause, C., Taentzer, G.: Henshin: advanced concepts and tools for in-place EMF model transformations. In: Petriu, D.C., Rouquette, N., Haugen, Ø. (eds.) Proceedings of the 13th International Conference on Model Driven Engineering Languages and Systems, MODELS’10, pp. 121–135. Springer, Berlin (2010)
Arendt, T., Habel, A., Radke, H., Taentzer, G.: From core OCL invariants to nested graph constraints. In: Giese, H., önig, B.K. (eds.) Proceedings of the 7th International Conference on Graph Transformation, ICGT’14, pp. 97–112. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09108-2_7
Bailey, C., Montrieux, L., de Lemos, R., Yu, Y., Wermelinger, M.: Run-time generation, transformation, and verification of access control models for self-protection. In: Proceedings of the 9th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2014, pp. 135–144. ACM, New York, NY, USA (2014). https://doi.org/10.1145/2593929.2593945
Brottier, E., Fleurey, F., Steel, J., Baudry, B., Le Traon, Y.: Metamodel-based test generation for model transformations: an algorithm and a tool. In: Proceedings of the 7th International Symposium on Software Reliability Engineering, ISSRE’06, pp. 85–94. IEEE (2006)
Cabot, J., Clarisó, R., Guerra, E., de Lara, J.: Verification and validation of declarative model-to-model transformations through invariants. J. Syst. Softw. 83(2), 283–302 (2010)
Chen, X., Li, A., Zeng, X., Guo, W., Huang, G.: Runtime model based approach to iot application development. Front. Comput. Sci. China 9(4), 540–553 (2015)
Czarnecki, K., Helsen, S.: Feature-based survey of model transformation approaches. IBM Syst. J. 45(3), 621–645 (2006). https://doi.org/10.1147/sj.453.0621
Ehrig, K., Küster, J.M., Taentzer, G.: Generating instance models from meta models. Softw. Syst. Model. 8(4), 479–500 (2009)
Finot, O., Mottu, J.M., Sunye, G., Attiogbe, C.: Partial test oracle in model transformation testing. In: Duddy, K., Kappel, G. (eds.) Proceedings of the 6th International Conference on Theory and Practice of Model Transformations, Lecture Notes in Computer Science, vol. 7909, pp. 189–204. Springer, Berlin (2013)
Fleurey, F., Baudry, B., Muller, P., Traon, Y.L.: Qualifying input test data for model transformations. Softw. Syst. Model. 8(2), 185–203 (2009)
Fürst, L., Mernik, M., Mahnič, V.: Converting metamodels to graph grammars: doing without advanced graph grammar features. Softw. Syst. Model. 14(3), 1297–1317 (2015)
Giese, H., Wagner, R.: From model transformation to incremental bidirectional model synchronization. Softw. Syst. Model. 8(1), 21–43 (2009)
Giner, P., Pelechano, V.: Test-driven development of model transformations. In: Schürr, A., Selic, B. (eds.) Proceedings of the 12th International Conference on Model Driven Engineering Languages and Systems, Lecture Notes in Computer Science, vol. 5795, pp. 748–752. Springer, Berlin (2009)
González, C.A., Cabot, J.: Atltest: a white-box test generation approach for ATL transformations. In: Proceedings of the 15th International Conference on Model Driven Engineering Languages and Systems, pp. 449–464. Springer (2012)
Guerra, E., Soeken, M.: Specification-driven model transformation testing. Softw. Syst. Model. 14(2), 623–644 (2015)
He, X., Fu, Y., Sun, C.A., Ma, Z., Shao, W.: Towards model-driven variability-based flexible service compositions. In: Proceedings of the 39th Annual Conference on Computer Software and Applications Conference, COMPSAC’15, vol. 2, pp. 298–303 (2015). https://doi.org/10.1109/COMPSAC.2015.64
He, X., Li, W., Zhang, T., Liu, Y.: Towards parallel model generation for random performance testing of model-oriented operations. In: Proceedings of the 10th International Symposium on Theoretical Aspects of Software Engineering, TASE’16, pp. 57–64 (2016)
He, X., Zhang, T., Hu, C.J., Ma, Z., Shao, W.: An MDE performance testing framework based on random model generation. J. Syst. Softw. 121, 247–264 (2016). https://doi.org/10.1016/j.jss.2016.04.044
He, X., Zhang, T., Ma, Z., Shao, W.: Randomized model generation for performance testing of model transformations. In: Proceedings of the 38th Annual International Computer Software and Applications Conference, COMPSAC’14. IEEE (2014)
Izsó, B., Szárnyas, G., Ráth, I.: Train benchmark. Budapest University of Technology and Economics, Technical report (2014)
Jackson, D., Schechter, I., Shlyakhter, I.: Alcoa: The alloy constraint analyzer. In: Proceedings of the International Conference on Software Engineering, ICSE’00, pp. 730–733. IEEE (2000)
Kolovos, D.S., Paige, R.F., Polack, F.A.C.: The epsilon object language (EOL). In: Rensink, A., Warmer, J. (eds.) Proceedings of the 2nd European Conference on Model Driven Architecture—Foundations and Applications, ECMDA-FA 2006, pp. 128–142. Springer, Berlin (2006). https://doi.org/10.1007/11787044_11
Koning, M., ai Sun, C., Sinnema, M., Avgeriou, P.: VxBPEL: supporting variability for web services in BPEL. Inf. Softw. Technol. 51(2), 258–269 (2009). https://doi.org/10.1016/j.infsof.2007.12.002
Kuhlmann, M., Hamann, L., Gogolla, M., Büttner, F.: A benchmark for OCL engine accuracy, determinateness, and efficiency. Softw. Syst. Model. 11(2), 165–182 (2012)
McQuillan, J.A., Power, J.F.: A metamodel for the measurement of object-oriented systems: An analysis using alloy. In: Proceedings of the 1st International Conference on Software Testing, Verification, and Validation, pp. 288–297. IEEE (2008)
Mougenot, A., Darrasse, A., Blanc, X., Soria, M.: Uniform random generation of huge metamodel instances. In: Proceedings of the 5th European Conference on Model Driven Architecture-Foundations and Applications, pp. 130–145. Springer (2009)
OASIS: Web services business process execution language version 2.0. Technical report, OASIS (2007)
Object Management Group: OMG Object Constraint Language (OCL) Specification Version 2.3.1 (2012). http://www.omg.org/spec/OCL/2.3.1
Pérez, C.A.G., Buettner, F., Clarisó, R., Cabot, J., et al.: Emftocsp: a tool for the lightweight verification of emf models. In: Proceedings of Formal Methods in Software Engineering: Rigorous and Agile Approaches (FormSERA) (2012)
Pietsch, P., Yazdi, H.S., Kelter, U.: Controlled generation of models with defined properties. In: Jähnichen, S., Küpper, A., Albayrak, S. (eds.) Software Engineering. Lecture Notes in Informatics, vol. 198, pp. 95–106 (2012)
Popoola, S., Kolovos, D.S., Rodriguez, H.H.: EMG: a domain-specific transformation language for synthetic model generation. In: Van Gorp, P., Engels, G. (eds.) Proceedings of the 9th International Conference on Theory and Practice of Model Transformations, ICMT 2016, pp. 36–51. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-42064-6_3
Radke, H.: A Theory of HR* Graph Conditions and their Application to Meta-Modeling. Phd. thesis, Carl von Ossietzky Universität Oldenburg (2016)
Radke, H., Arendt, T., Becker, J.S., Habel, A., Taentzer, G.: Translating essential OCL invariants to nested graph constraints focusing on set operations. In: Parisi-Presicce, F., Westfechtel, B. (eds.) Proceedings of the 8th International Conference on Graph Transformation, ICGT’15, pp. 155–170. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21145-9_10
Scheidgen, M.: Generation of large random models for benchmarking. In: Proceedings of the 3rd Workshop on Scalable Model Driven Engineering, pp. 1–10. L’Aquila, Italy (2015)
Sen, S., Baudry, B., Mottu, J.M.: On combining multi-formalism knowledge to select models for model transformation testing. Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, ICST ’08, pp. 328–337. IEEE Computer Society, Washington (2008). https://doi.org/10.1109/ICST.2008.62
Sen, S., Baudry, B., Mottu, J.M.: Automatic model generation strategies for model transformation testing. In: Paige, R.F. (ed.) Proceedings of the 2nd International Conference on Theory and Practice of Model Transformations, ICMT 2009, pp. 148–164. Springer, Berlin (2009). https://doi.org/10.1007/978-3-642-02408-5_11
Soeken, M., Wille, R., Drechsler, R.: Encoding OCL data types for sat-based verification of UML/OCL models. In: Tests and Proofs, pp. 152–170. Springer (2011)
Song, H., Huang, G., Chauvel, F., Xiong, Y., Hu, Z., Sun, Y., Mei, H.: Supporting runtime software architecture: a bidirectional-transformation-based approach. J. Syst. Softw. 84(5), 711–723 (2011)
Sun, C.A., Rossing, R., Sinnema, M., Bulanov, P., Aiello, M.: Modeling and managing the variability of web service-based systems. J. Syst. Softw. 83(3), 502–516 (2010). https://doi.org/10.1016/j.jss.2009.10.011
Szárnyas, G., Izsó, B., Ráth, I., Varró, D.: The Train Benchmark: Cross-technology performance evaluation of continuous model validation. Softw. Syst. Model. (2017). https://doi.org/10.1007/s10270-016-0571-8
Taentzer, G.: Instance generation from type graphs with arbitrary multiplicities. ECEASST 47, 1–15 (2012). https://doi.org/10.14279/tuj.eceasst.47.727
Ujhelyi, Z., Bergmann, G., Hegedüs, Á., Horváth, Á., Izsó, B., Ráth, I., Szatmári, Z., Varró, D.: EMF-incquery: an integrated development environment for live model queries. Sci. Comput. Program. 98, 80–99 (2015)
Van Amstel, M., Bosems, S., Kurtev, I., Pires, L.F.: Performance in model transformations: experiments with ATL and QVT. In: Proceedings of the 4th International Conference on Theory and Practice of Model Transformations, pp. 198–212. Springer (2011)
Acknowledgements
We would like to thank Dr. Zheng Cheng for his valuable discussion and comments on the paper. We also thank the anonymous reviewers for their review comments and revision suggestions.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Dr. Heiko Dörr.
This work was supported by the National Natural Science Foundation of China (Grant Nos. 61300009, 61472180, 61502228, and 61672046) and Jiangsu Province Research Foundation (Grant Nos. BK20141322, BK20150589).
Appendices
Appendix A: Well-definedness rules
This appendix lists the basic well-definedness rules of our template language. These rules are programmatically implemented and used to check whether a template model is syntactically well defined. These rules are expressed in a set of OCL invariants as follows:
Rule 1. A template cannot extend itself:
Given a Template, we define a query that collects all the TemplateNodes declared in this template or inherited from the super templates as follows:
Given a TemplateEdge, we define two queries that collect the source and the target TemplateNodes as follows:
Rule 2. A TemplateEdge must connect to the TemplateNodes that are defined in the same template:
Rule 3. If a TemplateEdge starts from a NestedNode, then a complete set of source Mappings must be defined. if a TemplateEdge ends with a NestedNode, then a complete set of target Mappings must be defined:
In Rule 3, the operation isComplete is defined as follows:
Rule 4. For a Mapping, the TemplateNode of the Mapping should be declared in the Template:
Rule 5. If we define a reversedNestedNode in a host Template, the referred templates of the reversedNestedNode must be the possible containers of the host Template:
Rule 6. A SimpleNode must refer to a concrete class:
Rule 6. The reference a TemplateEdge refers to must be consistent with the source and the target of the TemplateEdge:
Appendix B: Details of structured process model generation
1.1 B.1: Templates for structured process
The following templates were used to generate a structural process model (complying with UML activity diagram). From now on, the keyword nested can be omitted in the template definition for the sake of simplicity.
1.2 B.2: Wrapper metamodel for structured process
From the templates above, a wrapper metamodel was automatically derived as shown in Fig. 19.
1.3 B.3: Correctness
We present the proof of the correctness for the templates presented in Appendix B.1.
We consider a model, which conforms to UML activity diagram, is a process fragment iff it does not contain any InitialNodes and FinalNodes.
According to the wrapper metamodel in Fig. 19, any wrapper model conforming to the metamodel can be viewed as a tree, where the edges are containment relationships. The root node of the tree must be an instance of \(\mathbb {I}_{template}^{class}(TModel)\). The root node must contain exactly one child node that is an instance of \(\mathbb {I}_{template}^{class}(Tprocess)\). If we ignore the two nodes, the remainder of the wrapper model (namely the tailored wrapper model) is also a tree.
Now we prove that for any tailored wrapper model M, unfolding M will always result in a structured process fragment by using mathematic induction.
Assume that tm is the template model presented in Appendix B.1. Let |M| be the size of M.
When \(|M|=1\), M contains a single element that must be an instance of \(\mathbb {I}_{template}^{class}(TAction)\). According to the template TAction and the unfolding algorithms, the result of unfolding M is an CallAction, which is a structured process fragment.
We assume that when \(|M|\le k\), unfolding M results in a structured process fragment.
For the case \(|M|=k+1\), if we remove the root node \(n_r\) of |M|, we get a set of subtrees \(\{M_1,\ldots ,M_i\}\). According to Algorithm 7, \(\mathbb {U}_{\llbracket tm \rrbracket }(M)\) is equal to
which can be rewritten as
The size of \(M_j\) (\(1\le j\le i\)) is smaller than or equal with k. Unfolding \(\{M_1,\ldots ,M_i\}\) will result in a set of structured process fragments, according to the induction hypothesis.
Now consider unfolding \(n_r\). Depending on the type of \(n_r\), we have the following cases:
-
\(type_{\mathcal {N}}(n_r)=\mathbb {I}_{template}^{class}(TSequence)\) (Fig. 20a). This means \(n_r\) has two subtrees \(M_1\) and \(M_2\). Assume that unfolding \(M_1\) and \(M_2\) results in two structured process fragments \(f_1\) and \(f_2\). According to template TSequence, a ControlFlow will be created from the SimpleNodec (i.e., \(\mathbb {U}_{\llbracket c \rrbracket }^N(n_r)\)) during unfolding \(n_r\). Afterward, SimpleEdges e1 and e2 will be instantiated through connecting the ControlFlow to \(f_1\) and \(f_2\) (i.e., \(\mathbb {U}_{\llbracket e1 \rrbracket }^E(n_r)\) and \(\mathbb {U}_{\llbracket e2 \rrbracket }^E(n_r)\)). Since the remainder of the template TSequence (e.g., the reversedNestedNodeproc and the SimpleEdgee3) does not involve the constraint of a structured process, we can ignore them in this proof. According to the EndpointBindings associated with e1 and e2 and Algorithms 3 to 5, we know that the source of the ControlFlow will be connected to the exit node of \(f_1\), and the target of the ControlFlow will be connected to the entry node of \(f_2\). Thereby, \(f_1\) and \(f_2\) are strung together as a new sequence structure, namely \(f_{1+2}\). Obviously, \(f_{1+2}\), i.e., \(\mathbb {U}_{\llbracket \mathbb {I}^{-1}(type_{\mathcal {N}}(n_r) \rrbracket }^T(n_r)\), is also structured. The entry node of \(f_{1+2}\) is the entry node of \(f_1\), and the exit node of \(f_{1+2}\) is the exit node of \(f_2\).
-
\(type_{\mathcal {N}}(n_r)=\mathbb {I}_{template}^{class}(TChoice)\) (Fig. 20b). This means \(n_r\) has two subtrees \(M_1\) and \(M_2\). Assume that unfolding \(M_1\) and \(M_2\) results in two structured process fragments \(f_1\) and \(f_2\). According to template TChoice, during unfolding \(n_r\), a DecisionNode, a MergeNode, and four ControlFlows will be created from the SimpleNodes entry, exit, and c1 to c4 (i.e., \(\mathbb {U}_{\llbracket entry \rrbracket }^N(n_r)\), \(\mathbb {U}_{\llbracket exit \rrbracket }^N(n_r)\), and \(\mathbb {U}_{\llbracket c1 \rrbracket }^N(n_r)\) to \(\mathbb {U}_{\llbracket c4 \rrbracket }^N(n_r)\)), respectively. Then, the SimpleEdges e1 to e8 will be instantiated (i.e., \(\mathbb {U}_{\llbracket e1 \rrbracket }^E(n_r)\) to \(\mathbb {U}_{\llbracket e8 \rrbracket }^E(n_r)\)). Since the remainder of the template TChoice does not involve the constraint of a structured process, we can ignore them in this proof. According to the EndpointBindings associated with e2 and e4, and Algorithms 3 to 5, we can know that the target of the ControlFlowc1 will be connected to the entry node of \(f_1\), and the target of the ControlFlowc4 will be connected to the entry node of \(f_2\). Similarly, we can also know that the source of the ControlFlowc5 will be connected to the exit node of \(f_1\), and the source of the ControlFlowc7 will be connected to the exit node of \(f_2\). In this way, we construct a new choice structure (called \(f_{1+2}\)). Obviously, \(f_{1+2}\), i.e., \(\mathbb {U}_{\llbracket \mathbb {I}^{-1}(type_{\mathcal {N}}(n_r) \rrbracket }^T(n_r)\), is also structured. The entry node and the exit node of \(f_{1+2}\) are the DecisionNode and MergeNode newly created, respectively.
-
The cases of \(type_{\mathcal {N}}(n_r)=\mathbb {I}_{template}^{class}(TParallel)\) and \(type_{\mathcal {N}}(n_r)=\mathbb {I}_{template}^{class}(TLoop)\) are similar.
Therefore, the induction hypothesis also holds when \(|M|=k+1\). For any tailored wrapper model M, unfolding M always results in a structured process fragment.
After examining the template TProcess, we know that instantiating this template is to create an InitialNode and a FinalNode, and to create two ControlFlows from the InitialNode to the entry node of body and from the exit node of body to the FinalNode, respectively. The NestedNodebody actually points to the root node of the tailored wrapper model, which will be instantiated as a structured process fragment. Hence, the instance of the template TProcess is a structured process satisfying all the conditions mentioned in Sect. 6.1.
Appendix C: Details of train model generation
1.1 C.1 Templates for Train Model
The following templates were used to generate the train models. All the templates were derived from the source code of Szárnyas’s train model generator.
1.2 C.2 Correctness
As mentioned in Sect. 6.2, the train model has six well-formedness constraints. To evaluate the model queries that are used to check the six constraints, we must generate a train model containing both the fragments that satisfy the well-formedness constraints and the fragments that violate them.
Due to the space limitation, we analyze RouteSensor and SwitchSensor only.
Assume that the wrapper metamodel contains a fragment as illustrated in the left part of Fig. 21. Then, it is not difficult to find that unfolding this fragment results in a model fragment as illustrated in the right part of Fig. 21. Obviously, this resulting fragment satisfies both RouteSensor and SwitchSensor.
According to template TRoute, we know that element y, whose type is \(\mathbb {I}_{template}^{class}(TSwitch)\), in the wrapper model can be replaced by an element whose type is \(\mathbb {I}_{template}^{class}(TSwitchError)\). After doing this, unfolding the new fragment will lead to the absence of the relationship elements that currently exists in the right part of Fig. 21. It is because the template TSwitchError does not contain a TemplateEdge for creating this relationship (e.g., e1 in TSwitch). The new resulting model will violate the constraint SwitchSensor.
Similarly, if we replace the element z, whose type is \(\mathbb {I}_{template}^{class}(TSensor)\), in the wrapper model with an element whose type is \(\mathbb {I}_{template}^{class}(TSensorError1)\), the result of unfolding will violate the constraint RouteSensor. It is because the template TSensorError1 does not contain a TemplateEdge for creating the relationship definedBy.
We can ask the core generator to generate a wrapper model M that contains instances of \(\mathbb {I}_{template}^{class}(TSensor)\), \(\mathbb {I}_{template}^{class}(TSensorError)\), \(\mathbb {I}_{template}^{class}(TSensor)\) and \(\mathbb {I}_{template}^{class}(TSensorError1)\). Unfolding M will result in a model that contains the fragments satisfying RouteSensor and SwitchSensor and the fragments violating RouteSensor and/or SwitchSensor.
Appendix D: Details of RBAC model generation
The following templates were used to generate the RBAC model.
Note that to produce correct models, we must add two extra constraints in our core generator, as follows: (1) \(\{\mathbb {I}_{nested}^{reference}(u)\}\) is source-unique, where u is the NestedNode in WUserRole; (2) \(\{\mathbb {I}_{nested}^{reference}(r)\}\) is source-unique, where r is the NestedNode in WUserRole.
If a reference r is source-unique,
Please refer to our previous work [19,20,21] for more information about our core generator.
The correctness of the templates were explained in Section 7.
Appendix E: Details of statechart generation
The following templates were used to generate the statechart model.
Appendix F: Henshin rules
The section presents all the Henshin rules used in our case study.
To generate a structured process model using Henshin, we defined the five graph grammar rules in Fig. 22.
To generate a train model using Henshin, we defined the nine graph grammar rules in Fig. 23. Note that these rules can only produce a correct train model but cannot inject errors. If we want to inject errors, we will have to define more rules.
To generate a RBAC model using Henshin, we defined the five graph grammar rules in Fig. 24.
To generate a statechart model using Henshin, we defined the five graph grammar rules in Fig. 25.
Rights and permissions
About this article
Cite this article
He, X., Zhang, T., Pan, M. et al. Template-based model generation. Softw Syst Model 18, 2051–2092 (2019). https://doi.org/10.1007/s10270-017-0634-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10270-017-0634-5