Knowledge graph embedding in a uniform space
Abstract
Knowledge graph embedding (KGE) is typically used for link prediction to automatically predict missing links in knowledge graphs. Current KGE models are mainly based on complicated mathematical associations, which are highly expressive but ignore the uniformity behind the classical bilinear translational model TransE, a model that embeds all entities of knowledge graphs in a uniform space, enabling accurate embeddings. This study analyses the uniformity of TransE and proposes a novel KGE model called ConvUs that follows uniformity with expressiveness. Based on the convolution neural network (CNN), ConvUs proposes constraints on convolution filter values and employs a multi-layer, multi-scale CNN architecture with a non-parametric L2 norm-based scoring function for the calculation of triple scores. This addresses potential uniformity-related issues in existing CNN-based KGE models, allowing ConvUs to maintain a uniform embedding space while benefiting from the powerful expressiveness of CNNs. Furthermore, circular convolution is applied to alleviate the potential orderliness contradictions, making ConvUs more suitable for conducting uniform space KGE. Our model outperformed the base model ConvKB and several baselines on the link prediction benchmark WN18RR and FB15k-237, demonstrating strong applicability and generalization and indicating that the uniformity of embedding space with high expressiveness enables more efficient knowledge graph embeddings.
References
[1]
Y. Lan, G. He, J. Jiang, J. Jiang, W.X. Zhao and J.-R. Wen, A survey on complex knowledge base question answering: Methods, challenges and solutions, In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, 2021, pp. 4483–4491.
[2]
X. Chen, N. Zhang, L. Li, S. Deng, C. Tan, C. Xu, F. Huang, L. Si and H. Chen, Hybrid transformer with multi-level fusion for multimodal knowledge graph completion, In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, pp. 904–915.
[3]
H. Wang, Y. Zhang, Z. Ji, Y. Pang and L. Ma, Consensus-aware visual-semantic embedding for image-text matching, In Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIV, 2020, pp. 18–34.
[4]
A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston and O. Yakhnenko, Translating embeddings for modeling multi-relational data, Advances in Neural Information Processing Systems 26 (2013).
[5]
B. Yang, W.-T. Yih, X. He, J. Gao and L. Deng, Embedding entities and relations for learning and inference in knowledge bases, arXiv preprint arXiv:1412.6575, 2014.
[6]
T. Trouillon, J. Welbl, S. Riedel, É. Gaussier and G. Bouchard, Complex embeddings for simple link prediction, In International conference on machine learning, pp. 2071–2080. PMLR, 2016.
[7]
S.M. Kazemi and D. Poole, Simple embedding for link prediction in knowledge graphs, Advances in Neural Information Processing Systems 31 (2018).
[8]
W. Xu, S. Zheng, L. He, B. Shao, J. Yin and T.-Y. Liu, SEEK: Segmented embedding of knowledge graphs, In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 3888–3897.
[9]
D.Q. Nguyen, T.D. Nguyen, D.Q. Nguyen and D. Phung, A novel embedding model for knowledge base completion based on convolutional neural network, arXiv preprint arXiv:1712.02121, 2017.
[10]
T. Dettmers, P. Minervini, P. Stenetorp and S. Riedel, Convolutional 2D knowledge graph embeddings, In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
[11]
J. Huang, T. Zhang, J. Zhu, W. Yu, Y. Tang and Y. He, A deep embedding model for knowledge graph completion based on attention mechanism, 33(15) (2021), 9751–9760.
[12]
M. Schlichtkrull, N. Kipf, P. Bloem, R.V.D. Berg, I. Titov and M. Welling, Modeling relational data with graph convolutional networks, In European semantic web conference, Springer, 2018, pp. 593–607.
[13]
D. Nathani, J. Chauhan, C. Sharma and M. Kaul, Learning attention-based embeddings for relation prediction in knowledge graphs, In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 4710–4723.
[14]
R. Ye, X. Li, Y. Fang, H. Zang and M. Wang, A vectorized relational graph convolutional network for multi-relational network alignment, In Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019, pp. 4135–4141.
[15]
Z. Wang, J. Zhang, J. Feng and Z. Chen, Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI conference on artificial intelligence, volume 28, 2014.
[16]
Y. Lin, Z. Liu, M. Sun, Y. Liu and X. Zhu, Learning entity and relation embeddings for knowledge graph completion, In Twenty-ninth AAAI conference on artificial intelligence, 2015.
[17]
G. Ji, S. He, L. Xu, K. Liu and J. Zhao, Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: Long papers), 2015, pp. 687–696.
[18]
S. Yang, J. Tian, H. Zhang, J. Yan, H. He and Y. Jin, Transms: Knowledge graph embedding for complex relations by multidirectional semantics, In Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019, pp. 1935–1942.
[19]
W. Zhang, B. Paudel, W. Zhang, A. Bernstein and H. Chen, Interaction embeddings for prediction and explanation in knowledge graphs, 2019, pp. 96–104.
[20]
M. Nickel, V. Tresp and H.-P. Kriegel, A three-way model for collective learning on multi-relational data, In Icml, 2011.
[21]
C. Szegedy, S. Ioffe, V. Vanhoucke and A.A. Alemi, Inception-v4, inception-resnet and the impact of residual connections on learning, In Thirty-first AAAI conference on artificial intelligence, 2017.
[22]
K. Toutanova and D. Chen, Observed versus latent features for knowledge base and text inference, In Proceedings of the 3rd workshop on continuous vector space models and their compositionality, 2015, pp. 57–66.
[23]
K. He, X. Zhang, S. Ren and J. Sun, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, In Proceedings of the IEEE international conference on computer vision, 2015, pp. 1026–1034.
[24]
X. Han, S. Cao, X. Lv, Y. Lin, Z. Liu, M. Sun and J. Li, Openke: An open toolkit for knowledge embedding, In Proceedings of EMNLP, 2018, pp. 139–144.
Recommendations
Learning Embedding for Knowledge Graph Completion with Hypernetwork
Computational Collective IntelligenceAbstractLink prediction in Knowledge Graph, also called knowledge completion, is a significant problem in graph mining and has many applications for large companies. The more accurate the link prediction results will bring satisfaction, reduce and avoid ...
Effective Knowledge Graph Embedding with Quaternion Convolutional Networks
Natural Language Processing and Chinese ComputingAbstractConvolutional Neural Networks (CNNs) have demonstrated effectiveness in knowledge graph embedding, but existing CNN-based methods encounter two main challenges. Firstly, CNN-based models with simple architectures are unable to extract latent ...
Enhancing the convolution-based knowledge graph embeddings by increasing dimension-wise interactions
AbstractKnowledge graph embedding learns distributed low-dimensional representations for the elements in knowledge graphs, so that knowledge can be conveniently integrated into various tasks and smart systems. Recently, convolutional neural network has ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© 2024 – IOS Press. All rights reserved.
Publisher
IOS Press
Netherlands
Publication History
Published: 03 February 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Dec 2024