Abstract
Ontology embedding methods have been popular in recent years, especially when it comes to representation learning algorithms for solving ontology-related tasks. Despite the impact of large language models on knowledge graphs’ related tasks, there has been less focus on adapting these models to construct ontology embeddings that are both semantically relevant and faithful to the ontological structure. In this paper, we present a novel ontology embedding method that encodes ontology classes into a pre-trained SBERT through random walks and then fine-tunes the embeddings using a distance-based regression loss. We benchmark our algorithm on four different datasets across two tasks and show the impact of transfer learning and our distance-based loss on the quality of the embeddings. Our results show that SORBET outperform state-of-the-art ontology embedding techniques for the performed tasks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
http://geneontology.org/ontology/ accessed on the 2020-09-08.
- 5.
- 6.
- 7.
References
Ontology Matching 2021. Proceedings of the 16th International Workshop on Ontology Matching Co-located with the 20th International Semantic Web Conference (ISWC 2021), CEUR Workshop Proceedings, vol. 3063. CEUR-WS.org (2021)
Ontology Matching 2022. Proceedings of the 17th International Workshop on Ontology Matching (OM 2022) Co-located with the 21th International Semantic Web Conference (ISWC 2022), Hangzhou, Virtual Conference, 23 October 2022, CEUR Workshops Proceedings, vol. 3324. CEUR-WS.org (2022)
Ashburner, M., et al.: Gene ontology: tool for the unification of biology. Nat. Genet. 25(1), 25–29 (2000). https://doi.org/10.1038/75556
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Process. Syst. 26 (2013)
Chen, J., Hu, P., Jimenez-Ruiz, E., Holter, O.M., Antonyrajah, D., Horrocks, I.: OWL2Vec*: embedding of OWL ontologies. Mach. Learn. 110(7), 1813–1845 (2021). https://doi.org/10.1007/s10994-021-05997-6
Chen, J., Jiménez-Ruiz, E., Horrocks, I., Antonyrajah, D., Hadian, A., Lee, J.: Augmenting ontology alignment by semantic embedding and distant supervision. In: Verborgh, R., et al. (eds.) ESWC 2021. LNCS, vol. 12731, pp. 392–408. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77385-4_23
Consortium, T.G.O.: The gene ontology resource: enriching a gold mine. Nucl. Acids Res. 49(D1), D325–D334 (12 2020). https://doi.org/10.1093/nar/gkaa1113
Dooley, D.M., et al.: Foodon: a harmonized food ontology to increase global food traceability, quality control and data integration. NPJ Sci. Food 2(1), 23 (2018). https://doi.org/10.1038/s41538-018-0032-6
Efeoglu, S.: Graphmatcher: A graph representation learning approach for ontology matching. In: Ontology Matching 2022 : Proceedings of the 17th International Workshop on Ontology Matching (OM 2022) Co-located with the 21th International Semantic Web Conference (ISWC 2022), Hangzhou, Virtual Conference, 23 October 2022 [2], pp. 174–180 (2022)
Gosselin, F., Zouaq, A.: Sebmatcher results for OAEI 2022. In: Ontology Matching 2022: Proceedings of the 17th International Workshop on Ontology Matching (OM 2022) Co-located with the 21th International Semantic Web Conference (ISWC 2022), Hangzhou, Virtual Conference, 23 October 2022 [2], pp. 202–209 (2022)
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864 (2016)
Gutiérrez-Basulto, V., Schockaert, S.: From knowledge graph embedding to ontology embedding? an analysis of the compatibility between vector space representations and rules. In: International Conference on Principles of Knowledge Representation and Reasoning (2018)
He, Y., Chen, J., Antonyrajah, D., Horrocks, I.: Bertmap: a bert-based ontology alignment system. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 5684–5691 (2022)
Iyer, V., Agarwal, A., Kumar, H.: VeeAlign: multifaceted context representation using dual attention for ontology alignment. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. pp. 10780–10792. Association for Computational Linguistics, Punta Cana (2021). https://doi.org/10.18653/v1/2021.emnlp-main.842
Knorr, L., Portisch, J.: Fine-tom matcher results for OAEI 2021. In: Ontology Matching 2021: Proceedings of the 16th International Workshop on Ontology Matching Co-located with the 20th International Semantic Web Conference (ISWC 2021) [1], pp. 144–151 (2021)
Kolyvakis, P., Kalousis, A., Smith, B., Kiritsis, D.: Biomedical ontology alignment: an approach based on representation learning. J. Biomed. Semant. 9(1), 1–20 (2018)
Kossack, D., Borg, N., Knorr, L., Portisch, J.: Tom matcher results for OAEI 2021. In: Ontology Matching 2021: Proceedings of the 16th International Workshop on Ontology Matching co-located with the 20th International Semantic Web Conference (ISWC 2021) [1], pp. 193–198 (2021)
Kulmanov, M., Liu-Wei, W., Yan, Y., Hoehndorf, R.: El embeddings: geometric construction of models for the description logic el ++. Int. Joint Conf. Artif. Intell. (2019)
Li, C., Li, A., Wang, Y., Tu, H., Song, Y.: A survey on approaches and applications of knowledge representation learning. In: 2020 IEEE Fifth International Conference on Data Science in Cyberspace (DSC), pp. 312–319. IEEE (2020)
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 29 (2015)
Liu, W., et al.: K-bert: enabling language representation with knowledge graph. In: AAAI Conference on Artificial Intelligence (2019). https://api.semanticscholar.org/CorpusID:202583325
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, 2–4 May 2013, Workshop Track Proceedings (2013)
Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710 (2014)
Reimers, N., Gurevych, I.: Sentence-bert: sentence embeddings using siamese bert-networks. In: Conference on Empirical Methods in Natural Language Processing (2019)
Ristoski, P., Rosati, J., Di Noia, T., De Leone, R., Paulheim, H.: Rdf2vec: Rdf graph embeddings and their applications. Semant. Web 10(4), 721–752 (2019)
Smaili, F.Z., Gao, X., Hoehndorf, R.: Onto2vec: joint vector-based representation of biological entities and their ontology-based annotations. Bioinformatics 34(13), i52–i60 (2018)
Smaili, F.Z., Gao, X., Hoehndorf, R.: Opa2vec: combining formal and informal content of biomedical ontologies to improve similarity-based prediction. Bioinformatics 35(12), 2133–2140 (2019)
Sousa, G., Lima, R., Trojahn, C.: An eye on representation learning in ontology matching. In: Ontology Matching 2022: Proceedings of the 17th International Workshop on Ontology Matching (OM 2022) Co-located with the 21th International Semantic Web Conference (ISWC 2022), Hangzhou, Virtual Conference, October 23, 2022 [2], pp. 49–60 (2022)
Sun, T., et al.: Colake: contextualized language and knowledge embedding. arXiv preprint arXiv:2010.00309 (2020)
Wang, X., Gao, T., Zhu, Z., Liu, Z., Li, J.Z., Tang, J.: Kepler: a unified model for knowledge embedding and pre-trained language representation. Trans. Assoc. Comput. Linguist. 9, 176–194 (2019)
Wu, J., Lv, J., Guo, H., Ma, S.: Daeom: a deep attentional embedding approach for biomedical ontology matching. Appl. Sci. 10, 7909 (2020)
Yanardag, P., Vishwanathan, S.: Deep graph kernels. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1365–1374 (2015)
Acknowledgment
This research has been funded by the NSERC Discovery Grant Program. The authors acknowledge support from Compute Canada for providing computational resources.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gosselin, F., Zouaq, A. (2023). SORBET: A Siamese Network for Ontology Embeddings Using a Distance-Based Regression Loss and BERT. In: Payne, T.R., et al. The Semantic Web – ISWC 2023. ISWC 2023. Lecture Notes in Computer Science, vol 14265. Springer, Cham. https://doi.org/10.1007/978-3-031-47240-4_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-47240-4_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47239-8
Online ISBN: 978-3-031-47240-4
eBook Packages: Computer ScienceComputer Science (R0)