[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

TPmod: A Tendency-Guided Prediction Model for Temporal Knowledge Graph Completion

Published: 21 April 2021 Publication History

Abstract

Temporal knowledge graphs (TKGs) have become useful resources for numerous Artificial Intelligence applications, but they are far from completeness. Inferring missing events in temporal knowledge graphs is a fundamental and challenging task. However, most existing methods solely focus on entity features or consider the entities and relations in a disjoint manner. They do not integrate the features of entities and relations in their modeling process. In this paper, we propose TPmod, a tendency-guided prediction model, to predict the missing events for TKGs (extrapolation). Differing from existing works, we propose two definitions for TKGs: the Goodness of relations and the Closeness of entity pairs. More importantly, inspired by the attention mechanism, we propose a novel tendency strategy to guide our aggregated process. It integrates the features of entities and relations, and assigns varying weights to different past events. What is more, we select the Gate Recurrent Unit (GRU) as our sequential encoder to model the temporal dependency in TKGs. Besides, the Softmax function is employed to generate the final decreasing group of candidate entities. We evaluate our model on two TKG datasets: GDELT-5 and ICEWS-250. Experimental results show that our method has a significant and consistent improvement compared to state-of-the-art baselines.

References

[1]
Giang Hoang Nguyen, Jhon Boaz Lee, Ryan A. Rossi, Nesreen K. Ahmed, Eunyee Koh, and Sungchul Kim. 2018. Continuous-time dynamic network embeddings. In Companion Proceedings of the The Web Conference 2018. 969–976.
[2]
Caiming Xiong, Stephen Merity, and Richard Scoher. 2016. Dynamic memory networks for visual and textual question answering. In Proceedings of the International Conference on Machine Learning. 2397–2406.
[3]
Caiming Xiong, Victor Zhong, and Richaid Socher. 2017. Dynamic coattention networks for question answering. In Proceedings of the International Conference on Learning Representations.
[4]
Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. 2019. DyRep: Learning representations over dynamic graphs. In Proceedings of the International Conference on Learning Representations.
[5]
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, 1247–1250.
[6]
George A. Miller. 1995. WordNet: A lexical database for English. Communications of the ACM 38, 11 (1995), 39–41.
[7]
Zhiqing Sun, Zhihong Deng, Jianyun Nie, and Jian Tang. 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proceedings of the International Conference on Learning Representations.
[8]
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Eric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the 33rd International Conference on Machine Learning. 2071–2080.
[9]
Seyed Mehran Kazemi and David Poole. 2018. SimplE embedding for link prediction in knowledge graphs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 4289–4300.
[10]
Kalev Leetaru and Philip A. Schrodt. 2013. GDELT: Global data on events, location, and tone, 1979–2012. In Proceedings of the International Studies Association Annual Convention.
[11]
Elizabeth Boschee, Janin Lautenschläger, Sean O'Brien, Stephen M. Shellman, James Starz, and Michael D. Ward. 2015. ICEWS Coded Event Data. Harvard Dataverse 12. Retrieved from https://doi.org/10.7910/DVN/28075.
[12]
Philip A. Schrodt. 2012. CAMEO: Conflict and Mediation Event Observations Event and Actor Codebook. Pennsylvania State University, University Park, PA.
[13]
Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. 2017. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In Proceedings of the 34th Proceedings of the International Conference on Machine Learning. Vol. 70. ACM, 3462–3471.
[14]
Woojeong Jin, Changlin Zhang, Pedro Szekely, and Xiang Ren. 2019. Recurrent event network for reasoning over temporal knowledge graphs. In Proceedings of the 7th International Conference on Learning Representations.
[15]
Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge graph embedding: a survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering 29, 12 (2017), 2724–2743.
[16]
Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and Philip S. Yu. 2020. A survey on knowledge graphs: Representation, acquisition and applications. arXiv:2002.00388. Retrieved from https://arxiv.org/abs/2002.00388.
[17]
Bianchi Federico, Gaetano Rossiello, Luca Costabello, Matteo Palmonari, and Pasquale Minervini. 2020. Knowledge graph embeddings and explainable AI. arXiv:2004.14843. Retrieved from https://arxiv.org/abs/2004.14843.
[18]
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems. Vol. 2. ACM, 2787–2795.
[19]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 1112–1119.
[20]
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 2181–2187.
[21]
Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2016. Representation learning of knowledge graphs with hierarchical types. In Proceedings of the International Joint Conference on Artificial Intelligence. 2965–2971.
[22]
Ruobing Xie, Zhiyuan Liu, Jia Jia, Hanbo Luan, and Maosong Sun. 2016. Representation learning of knowledge graphs with entity descriptions. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 189–205.
[23]
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th Proceedings of the International Conference on Machine Learning. 809–816.
[24]
Bishan Yang, Scott Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the International Conference on Learning Representations.
[25]
Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. 2016. Holographic embeddings of knowledge graphs. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 1955–1961.
[26]
Ivana Balažević, Carl Allen, and Timothy M. Hospedales. 2019. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the International Joint Conference on Natural Language Processing.
[27]
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2D knowledge graph embeddings. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence.
[28]
Ivana Balazevic, Carl Allen, and Timothy M. Hospedales. 2019. Hypernetwork knowledge graph embeddings. In Proceedings of the International Conference on Artificial Neural Networks. I. Tetko, V. Kůrková, P. Karpov, F. Theis (Eds.), Lecture Notes in Computer Science, Vol. 11731, Springer, Cham, 553–565.
[29]
Julien Leblay and Melisachew Wudage Chekol. 2018. Deriving validity time in knowledge graph. In Companion Proceedings of the The Web Conference 2018. 1771–1776.
[30]
Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha Talukdar. 2018. HyTE: Hyperplane-based temporally aware knowledge graph embedding. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2001–2011.
[31]
Tingsong Jiang, Tianyu Liu, Tao Ge, Lei Sha, Sujian Li, Baobao Chang, and Zhifang Sui. 2016. Encoding temporal information for time-aware link prediction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2350–2354.
[32]
Rishab Goel, Seyed Mehran Kazemi, Marcus Brubaker, and Pascal Poupart. 2020. Diachronic embedding for temporal knowledge graph completion. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 3988–3995
[33]
Yunpu Ma, Volker Tresp, and Erik A. Daxberger. 2018. Embedding models for episodic knowledge graphs. Journal of Web Semantics. arXiv:1807.00228. Retrieved from https://arxiv.org/abs/1807.00228.
[34]
Timoth´ee Lacroix, Guillaume Obozinski, and Nicolas Usunier. 2020. Tensor decompositions for temporal knowledge base completion. In Proceedings of the International Conference on Learning Representations.
[35]
Alberto García-Durán, Sebastijan Dumančić, and Mathias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 4816–4821.
[36]
Seyed Mehran Kazemi, Rishab Goel, Kshitij Jain, Ivan Kobyzev, Akshay Sethi, Peter Forsyth, and Pascal Poupart. 2020. Relational representation learning for dynamic (knowledge) graphs: A survey. arXiv:1905.11485. Retrieved from https://arxiv.org/pdf/1905.11485v1.pdf.
[37]
Veličković Petar, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. In Proceedings of the International Conference on Learning Representations.
[38]
Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. 2020. DySAT: Deep neural representation learning on dynamic graphs via self-attention networks. In Proceedings of the 13th International Conference on Web Search and Data Mining. 519–527.
[39]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st Conference on Neural Information Processing Systems.
[40]
Kyunghyun Cho, Bart van Merrienboer, C¸ aglar G¨ulc¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 5998–6008.
[41]
Srivastava, Nitish, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1 (2014), 1929–1958.
[42]
Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from https://arxiv.org/abs/1412.6980.
[43]
Ruffinelli Daniel, Samuel Broscheit, and Rainer Gemulla. 2019. You CAN teach an old dog new tricks! On training knowledge graph embeddings. In Proceedings of the International Conference on Learning Representations.
[44]
Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. 2013. YAGO3: A knowledge base from multilingual wikipedias. In Proceedings of the 7th Biennial Conference on Innovative Data Systems Research.
[45]
Fredo Erxleben, Michael Günther, Markus Krötzsch, Julian Mendez, and Denny Vrandeăić. 2014. Introducing wikidata to the linked data web. In Proceedings of the 13th International Semantic Web Conference. 50–65.

Cited By

View all
  • (2024)Intricate Spatiotemporal Dependency Learning for Temporal Knowledge Graph ReasoningACM Transactions on Knowledge Discovery from Data10.1145/364836618:6(1-19)Online publication date: 12-Apr-2024
  • (2024)Multi-hop temporal knowledge graph reasoning with multi-agent reinforcement learningApplied Soft Computing10.1016/j.asoc.2024.111727160:COnline publication date: 1-Jul-2024
  • (2024)Temporal inductive path neural network for temporal knowledge graph reasoningArtificial Intelligence10.1016/j.artint.2024.104085329:COnline publication date: 1-Apr-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data
ACM Transactions on Knowledge Discovery from Data  Volume 15, Issue 3
June 2021
533 pages
ISSN:1556-4681
EISSN:1556-472X
DOI:10.1145/3454120
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2021
Accepted: 01 December 2020
Revised: 01 November 2020
Received: 01 July 2020
Published in TKDD Volume 15, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Temporal knowledge graph
  2. completion
  3. tendency strategy

Qualifiers

  • Research-article
  • Refereed

Funding Sources

  • National Natural Science Foundation of China
  • Natural Science Foundation of Hebei Province
  • Natural Science Foundation of Liaoning Province
  • Key Project of Scientific Research Funds in Colleges and Universities of Hebei Education Department
  • Fundamental Research Funds for the Central Universities
  • Program for 333 Talents in Hebei Province

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)96
  • Downloads (Last 6 weeks)12
Reflects downloads up to 12 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Intricate Spatiotemporal Dependency Learning for Temporal Knowledge Graph ReasoningACM Transactions on Knowledge Discovery from Data10.1145/364836618:6(1-19)Online publication date: 12-Apr-2024
  • (2024)Multi-hop temporal knowledge graph reasoning with multi-agent reinforcement learningApplied Soft Computing10.1016/j.asoc.2024.111727160:COnline publication date: 1-Jul-2024
  • (2024)Temporal inductive path neural network for temporal knowledge graph reasoningArtificial Intelligence10.1016/j.artint.2024.104085329:COnline publication date: 1-Apr-2024
  • (2023)Sequence-Based Modeling for Temporal Knowledge Graph Link PredictionArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44216-2_45(550-562)Online publication date: 26-Sep-2023
  • (2022)Hyperplane-based time-aware knowledge graph embedding for temporal knowledge graph completionJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-21195042:6(5457-5469)Online publication date: 1-Jan-2022
  • (2022)Multi-Concept Representation Learning for Knowledge Graph CompletionACM Transactions on Knowledge Discovery from Data10.1145/353301717:1(1-19)Online publication date: 30-Apr-2022
  • (2021)Neighborhood aggregation based graph attention networks for open-world knowledge graph reasoningJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-21188941:2(3797-3808)Online publication date: 1-Jan-2021

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media