[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

A framework for qualitative assessment of domain-specific languages

  • Regular Paper
  • Published:
Software & Systems Modeling Aims and scope Submit manuscript

Abstract

Domain-specific languages (DSLs) are used for improving many facets of software development, but whether and to what extent this aim is achieved is an important issue that must be addressed. This paper presents a proposal for a Framework for Qualitative Assessment of DSLs (FQAD). FQAD is used for determining the perspective of the evaluator, understanding the goal of the assessment and selecting fundamental DSL quality characteristics to guide the evaluator in the process. This framework adapts and integrates the ISO/IEC 25010:2011 standard, CMMI maturity level evaluation approach and the scaling approach used in DESMET into a perspective-based assessment. A detailed list of domain-specific language quality characteristics is elaborated, and a novel assessment method is proposed. Two case studies through which FQAD is matured and evaluated are reported. The case studies have shown that stakeholders find the FQAD process beneficial.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Amstel, M.F., Lange, C.F., Brand, M.G.: Using Metrics for Assessing the Quality of ASF+SDF Model Transformations, ICMT. Springer, Berlin (2009)

    Google Scholar 

  2. Bunge, M.A., Ontology, I.: The Furniture of the World (vol. 3). Reidel, Dordrecht (1977)

  3. Cameron, K.S.: Critical questions in assessing organizational effectiveness. Organ. Dyn. 9(2), 66–80 (1980)

    Article  Google Scholar 

  4. Chen, Y., Dios, R., Mili, A., Wu, L., An Empirical Study of Programming Language Trends. New Jersey Institute of Technology, Kefei Wang, State University of New York, Albany, IEEE (2005)

  5. Frank, U.: Domain-specific modeling languages-requirements analysis and design guidelines. In: Reinhartz-Berger, I., Sturm, A., Clark, T., Wand, Y., Cohen, S., Bettin, J. (eds.) Domain Engineering: Product Lines, Conceptual Models, and Languages. Springer, Berlin (2013)

  6. Gabriel, P.H.N.: Software Langauges Engineering: Experimental Evaluation. Dissertacao apresentada na Faculdade Ciencias e Tecnologia da Universidade Nova de Lisboa para obtencao do grau de Mestre em Engenharia Informatica, Lisboa (2010)

  7. Haugen, O., Mohagheghi, P.: A multi-dimensional framework for characterizing domain specific languages. In: Proceeding of the 7th OOPSLA Workshop on Domain Specific Modeling (2007)

  8. Hermans, F., Pinzger, M., Deursen, A.V.: Domain specific langages in practice: a user study on the success factors. In: MODELS09, pp. 423–437. Springer, Berlin (2009)

  9. ISO/IEC 25010:2011: Systems and Software Engineering Systems and Software Quality Requirements and Evaluation (SQuaRE) System and Software Quality Models, International Standards Organization/International Electrotechnical Commission (2011)

  10. Kahlaoui, A., Abran, A., Lefebvre, E.: DSML success factors and their assessment criteria. Metrics News 13(1), 43–51 (2008)

    Google Scholar 

  11. Kärna, J., Tolvanen, J.P., Kelly, S.: Evaluating the use of domain-specific modeling in practice. In: Proceedings of DSM09 (2009)

  12. Karsai, G., Krahn, H., Pinkernell, C., Rumpe, B., Schindler, M., Volkel, S.: Design guidelines for domain specific languages. In: Proceedings of the 9th OOPSLA Workshop on Domain-Specific Modeling (DSM’ 09), Orlando, FL, USA (October 2009)

  13. Kelly, S., Tolvanen, J.P.: Domain Specific Modeling Enabling Full Code Generation. Wiley, New York (2008)

  14. Kelly, S., Pohjonen, R.: Worst practices for domain-specific modeling. IEEE Softw. 26(4), 22–29, (July/August 2009)

  15. Khedker, U.P.: What Makes a Good Programming Language? Technical Report TR-97-upk-1, Department of Computer Science University of Pune (1997)

  16. Kitchenham, B.A., Linkman, S., Law, D.: DESMET: a methodology for evaluating software engineering methods and tools. Comput. Control Eng. J. 8(3), 120–126 (1997)

    Google Scholar 

  17. Kolovos, D.S., Paige, R.F., Kelly, T.P., Polack, F.A.C.: Requirements for domain-specific languages. In: Proceedings of the First ECOOP Workshop on Domain-Specific Program Development, Co-located with ECOOP06, Nantes, France (2006)

  18. Kosar, T., Oliviera, N., Mernik, M., Pereira, V.M.J., Crepinsek, M., Cruz, D., Henriques, P.R.: Comparing general purpose and domain specific languages: an empirical study. ComSIS 7(2), 247–264 (Special Issue) (2010)

    Google Scholar 

  19. Kouhen, A.E., Dumoulin, C., Gerard, S., Boulet, P.: Evaluation of Modeling Tools Adaptation, hal-00706701, version 2 (2012)

  20. Krogstie, J.: Evaluating UML Using a Generic Quality Framework Chapter in UML and the Unified Process. Idea Group Publishing, Hershey (2003)

    Google Scholar 

  21. Merilinna, J., Pärssinen, J.: Comparison between different abstraction level programming: experiment definition and initial results. In: OOPSLA Workshop on Domain-Specific Modeling, Montreal, Canada (2007)

  22. McKean, D., Sprinkle, J.: Heterogeneous multi-core systems: UML profiles vs. DSM Approaches. In: Proceedings of the 2012 Workshop on Domain Specific Modeling (2012)

  23. Mohagheghi, P., Dehlen, V., Neple, T.: Definitions and approaches to model quality in model based software development a review of literature. Inf. Softw. Technol. 51, 1646–1669 (2009)

    Article  Google Scholar 

  24. Moody, D.L.: The physics of notations: toward a scientific basis for constructing visual notations in software engineering. IEEE Trans. Softw. Eng. 35, 756–779 (2009)

    Article  Google Scholar 

  25. Paige, R., Ostroff, J., Brooke, P.: Principles for Modeling Language Design, Techical Report CS-1999-08, York University (1999)

  26. Pfleeger, S.L., Kitchenham, B.A.: Principles of survey research. ACM SIGSOFT Softw. Eng. Notes 26, 16–18 (2001)

    Article  Google Scholar 

  27. Runeson, P., Host, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14–2, 131–164 (2009)

    Article  Google Scholar 

  28. Software Engineering Institute, CMMI for Acquisition, Version 1.3, Technical Report, CMU/SEI-2010-TR-032, Carnegie Mellon (November 2012)

  29. Strembeck, M., Zdun, U.: An approach for the systematic development of domain specific languages. Softw. Pract. Exp. 39, 1253–1292 (2009); John Wiley & Sons Ltd

    Google Scholar 

  30. Wand, Y., Weber, R.: On the ontological expressiveness of information systems analysis and design grammars. J. Inf. Syst. 3, 217–237 (1993)

    Article  Google Scholar 

  31. Wand, Y., Weber, R.: On the deep structure of information systems. Inf. Syst. J. 5, 203–223 (1995)

    Article  Google Scholar 

  32. Wu, Y., Hernandez, F., Ortega, F., Clarke, P.J., France, R.: Measuring the effort for creating and using domain specific models. In: Proceedings of 10th Domain Specific Modeling Workshop (2010)

Download references

Acknowledgments

We would like to thank especially the members of the ASELSAN Company REHIS and SST Group software development departments, where we conducted the case studies. The authors are deeply grateful for the thorough evaluation and constructive criticism provided by the anonymous reviewers whose suggestions have been crucial in improving the quality of this paper.

Author information

Authors and Affiliations

  1. Department of Electrical and Electronics Engineering, Middle East Technical University, Ankara, Turkey

    Gökhan Kahraman & Semih Bilgen

Authors
  1. Gökhan Kahraman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Semih Bilgen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gökhan Kahraman.

Additional information

Communicated by Dr. Oystein Haugen.

Appendices

Appendix: Assessment levels in FQAD

1.1 DSL quality sub-characteristics support levels

Sub-characteristics support levels apply to a DSLs quality achievement in individual characteristics. These levels are a means for understanding the state of the quality corresponding to a given characteristic. Support levels adapted from [16] and depicted in Table 7 are designated in an ordinal scale in which “full support” corresponds to the highest level. According to Kitchenham et al. [16], granularity of the support levels depends upon the feature that is assessed and requirements. Based on this, we defined four support levels for DSLs.

1.2 DSL success levels

Success levels apply to a DSLs quality achievement. These levels are a means of assessing a DSL. The three success levels are presented in Table 8.

Table 7 DSL quality sub-characteristics support levels
Full size table
Table 8 DSL success levels
Full size table

Form I

See Table 9.

Table 9 Characteristics importance ranking form
Full size table

Form II

See Table 10.

Table 10 Sub-characteristic assessment statements form
Full size table

Form III

See Table 11.

Table 11 Results are generated automatically in form III
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kahraman, G., Bilgen, S. A framework for qualitative assessment of domain-specific languages. Softw Syst Model 14, 1505–1526 (2015). https://doi.org/10.1007/s10270-013-0387-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-013-0387-8

Keywords

Search

Navigation