Abstract
We present a comprehensive approach to ontology evaluation and validation, which have become a crucial problem for the development of semantic technologies. Existing evaluation methods are integrated into one sigle framework by means of a formal model. This model consists, firstly, of a meta-ontology called O 2, that characterises ontologies as semiotic objects. Based on O 2 and an analysis of existing methodologies, we identify three main types of measures for evaluation: structural measures, that are typical of ontologies represented as graphs; functional measures, that are related to the intended use of an ontology and of its components; and usability-profiling measures, that depend on the level of annotation of the considered ontology. The meta-ontology is then complemented with an ontology of ontology validation called oQual, which provides the means to devise the best set of criteria for choosing an ontology over others in the context of a given project. Finally, we provide a small example of how to apply oQual-derived criteria to a validation case.
Chapter PDF
Similar content being viewed by others
Keywords
- Description Logic
- Semantic Technology
- Intended Conceptualization
- Ontology Description
- Competency Question
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Almuhareb, A., Poesio, M.: Attribute-based and value-based clustering: an evaluation. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (2004)
Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison Wesley, Reading (1999)
Berardi, D., Calvanese, D., De Giacomo, G.: Reasoning on UML Class Diagrams using Description Logic Based Systems. In: Proceedings of the KI, Workshop on Applications of Description Logics (2001)
Brewster, C., Alani, H., Dasmahapatra, S., Wilks, Y.: Data-driven ontology evaluation. In: Proceedings of LREC (2004)
Ciaramita, M., Gangemi, A., Ratsch, E., Saric, J., Rojas, I.: Unsupervised Learning of Semantic Relations between Concepts of a Molecular Biology Ontology. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (2005)
Daelemans, W., Reinberger, M.L.: Shallow Text Understanding for Ontology Content Evaluation. IEEE Intelligent Systems, 1541–1672 (2004)
Gangemi, A.: Ontology Design Patterns for Semantic Web Content. In: Motta, E., Gil, Y. (eds.) Proceedings of the Fourth International Semantic Web Conference (2005)
Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J.: Ontology evaluation: A review of methods and an integrated model for the quality diagnostic task. Technical Report (2005), Available at: http://www.loa-cnr.it/Publications.html
Gómez-Pérez, A.: Ontology Evaluation. In: Staab, S., Studer, R. (eds.) Handbook on Ontologies, pp. 251–274. Springer, Heidelberg (2003)
Guarino, N.: Towards a Formal Evaluation of Ontology Quality. IEEE Intelligent Systems, 1541–1672 (2004)
Hartmann, J., Spyns, P., Giboin, A., Maynard, D., Cuel, R., Suárez-Figueroa, M.C., Sure, Y.: Methods for ontology evaluation. Knowledge Web Deliverable D1.2.3 (2004)
Hartmann, J., Palma, R., Sure, Y., Suárez-Figueroa, M.C., Haase P.: OMV– Ontology Metadata Vocabulary. In: The Ontology Patterns for the Semantic Web (OPSW) Workshop at ISWC 2005, Galway, Ireland (2005), http://www.research.ibm.com/people/w/welty/OPSW-05/
Kaakinen, J., Hyona, J., Keenan, J.M.: Individual differences in perspective effects on on-line text processing. Discourse Processes 33, 159–173 (2002)
Lozano-Tello, A., Gómez-Pérez, A.: ONTOMETRIC: A method to choose the appropriate ontology. Journal of Database Management 15(2) (2004)
Masolo, C., Gangemi, A., Guarino, N., Oltramari, A., Schneider, L.: WonderWeb Deliverable D18: The WonderWeb Library of Foundational Ontologies (2004), Available at: http://www.loa-cnr.it/Publications.html
Noy, N.: Evaluation by Ontology Consumers. IEEE Intelligent Systems,1541–1672 (2004)
Peirce, C.S.: Collected Papers. Hartshorne, C., Weiss, P., Burks, A.W. (eds.), vol. 1-8. Harvard University Press, Cambridge (1931-1958)
Porzel, R., Malaka, R.: A Task-based Approach for Ontology Evaluation. In: Proceedings of ECAI 2004 (2004)
Spyns, P.: EvaLexon: Assessing triples mined from texts. Technical Report 09, STAR Lab, Brussel (2005)
Steels, L.: Components of Expertise. AI Magazine 11, 2, 30–49 (1990)
Sure, Y. (ed.): Why Evaluate Ontology Technologies? Because It Works. IEEE Intelligent Systems,1541–1672 (2004)
Uschold, U., Gruninger, M.: Ontologies: Principles, Methods, and Applications. Knowledge Eng. Rev. 11(2), 93–155 (1996)
Welty, C., Guarino, N.: Supporting ontological analysis of taxonomic relationships. Data and Knowledge Engineering 39(1), 51–74 (2001)
Yao, H., Orme, A.M., Etzkorn, L.: Cohesion Metrics for Ontology Design and Application. Journal of Computer Science 1(1), 107–113 (2005)
http://www.loa.cnr/ontologies/DLP_397.owl , http://www.dolce.semantic.web.org
http://oyster.ontoware.org , http://www.onthology.org , http://smi-protege.stanford.edu:8080/KnowledgeZone/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J. (2006). Modelling Ontology Evaluation and Validation. In: Sure, Y., Domingue, J. (eds) The Semantic Web: Research and Applications. ESWC 2006. Lecture Notes in Computer Science, vol 4011. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11762256_13
Download citation
DOI: https://doi.org/10.1007/11762256_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34544-2
Online ISBN: 978-3-540-34545-9
eBook Packages: Computer ScienceComputer Science (R0)