Attentional Neural Integral Equation for Temporal Knowledge Graph Forecasting
Pages 4128 - 4132
Abstract
Temporal Knowledge Graph Forecasting (TKGF) aims to forecast the missing entities or relations at a specific timestamp when only the historical information is observed. It is crucial to accurately identify the historical information of complex temporal relational graphs related to the query. Existing works, e.g., TANGO, have exploited the Neural Ordinary Differential Equation (NODE) to TKGF. However, TANGO encounters two limitations. First, TANGO observes historical facts with only one timestamp at each step, leading to a long-term forgetting problem. Second, TANGO gives the same weight to the entire history graph, including facts that are not relevant to the query. To tackle the above limitations, this paper utilizes Attentional Neural Integral Equation for TKGF (tIE), enabling the global interaction between query-related historical graph sequences. To achieve this, we employ the Relational Graph Convolutional Network and Fourier-type Transformer to model the graph structure and temporal evolution of TKG. The Iterative Integral Equation Solver is exploited to enhance the accuracy and robustness of numerical solutions. The proposed method outperforms baseline models regarding several metrics and inference speed on four benchmark datasets, especially on the long horizontal link forecasting task with irregular time intervals.
References
[1]
Ivana Balazevic, Carl Allen, and Timothy M. Hospedales. 2019. Tucker: tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3--7, 2019. Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, (Eds.) Association for Computational Linguistics, 5184--5193.
[2]
Shuhao Cao. 2021. Choose a transformer: fourier or galerkin. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6--14, 2021, virtual. Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, (Eds.), 24924--24940. https://proceedings.neurips.c c/paper/2021/hash/d0921d442ee91b896ad95059d13df618-Abstract.html.
[3]
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. 2018. Neural ordinary differential equations. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3--8, 2018, Montréal, Canada. Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, (Eds.), 6572--6583. https://proceedings.neurips.cc/paper/2018 /hash/69386f6bb1dfed68692a24c8686939b9-Abstract.html.
[4]
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2--7, 2018. Sheila A. McIlraith and Kilian Q. Weinberger, (Eds.) AAAI Press, 1811--1818. https://www.aaai.org/ocs/index.ph p/AAAI/AAAI18/paper/view/17366.
[5]
Alberto García-Durán, Sebastijan Dumancic, and Mathias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. Ellen Riloff, David Chiang, Julia Hockenmaier, (Eds.) Association for Computational Linguistics, 4816--4821. https://aclanthology.org/D18--1516/.
[6]
Claudio A. Gelmi and Héctor Jorquera. 2014. IDSOLVER: A general purpose solver for nth-order integro-differential equations. Comput. Phys. Commun., 185, 1, 392--397.
[7]
Rishab Goel, Seyed Mehran Kazemi, Marcus A. Brubaker, and Pascal Poupart. 2020. Diachronic embedding for temporal knowledge graph completion. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7--12, 2020. AAAI Press, 3988--3995.
[8]
Zhen Han, Zifeng Ding, Yunpu Ma, Yujia Gu, and Volker Tresp. 2021. Learning neural ordinary equations for forecasting future links on temporal knowledge graphs. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7--11 November, 2021. Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and ScottWen-tau Yih, (Eds.) Association for Computational Linguistics, 8352--8364.
[9]
Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. 2020. Recurrent event network: autoregressive structure inferenceover temporal knowledge graphs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16--20, 2020. Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, (Eds.) Association for Computational Linguistics, 6669--6683.
[10]
Timothée Lacroix, Guillaume Obozinski, and Nicolas Usunier. 2020. Tensor decompositions for temporal knowledge base completion. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26--30, 2020. OpenReview.net. https://openreview.net/forum?id=rke2P1BFwS.
[11]
Jennifer Lautenschlager, Steve Shellman, and Michael Ward. 2015. ICEWS Event Aggregations. Version V3.
[12]
Julien Leblay and Melisachew Wudage Chekol. 2018. Deriving validity time in knowledge graph. In Companion of the The Web Conference 2018 on The Web Conference 2018, WWW 2018, Lyon, France, April 23--27, 2018. Pierre-Antoine Champin, Fabien Gandon, Mounia Lalmas, and Panagiotis G. Ipeirotis, (Eds.) ACM, 1771--1776.
[13]
Zixuan Li, Xiaolong Jin, Wei Li, Saiping Guan, Jiafeng Guo, Huawei Shen, YuanzhuoWang, and Xueqi Cheng. 2021. Temporal knowledge graph reasoning based on evolutional representation learning. In SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11--15, 2021. Fernando Diaz, Chirag Shah, Torsten Suel, Pablo Castells, Rosie Jones, and Tetsuya Sakai, (Eds.) ACM, 408--417.
[14]
Fabian M. Suchanek, Gjergji Kasneci, and GerhardWeikum. 2007. Yago: A Core of Semantic Knowledge. In 16th International Conference on the World Wide Web, 697--706.
[15]
Haohai Sun, Shangyi Geng, Jialun Zhong, Han Hu, and Kun He. 2022. Graph hawkes transformer for extrapolated reasoning on temporal knowledge graphs. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7--11, 2022. Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, (Eds.) Association for Computational Linguistics, 7481--7493.
[16]
Haohai Sun, Jialun Zhong, Yunpu Ma, Zhen Han, and Kun He. 2021. Timetraveler: reinforcement learning for temporal knowledge graph forecasting. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7--11 November, 2021. Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and ScottWen-tau Yih, (Eds.) Association for Computational Linguistics, 8306--8319.
[17]
Théo Trouillon, JohannesWelbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19--24, 2016 (JMLR Workshop and Conference Proceedings). Maria-Florina Balcan and Kilian Q. Weinberger, (Eds.) Vol. 48. JMLR.org, 2071--2080. http://proceedings.mlr.press/v48/trouillon16.html.
[18]
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. 2020. Composition-based multi-relational graph convolutional networks. (2020). htt ps://arxiv.org/abs/1911.03082 arXiv: 1911.03082 [cs.LG].
[19]
Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, and Vikas Singh. 2021. Nyströmformer: A nyström-based algorithm for approximating self-attention. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2--9, 2021. AAAI Press, 14138--14148.
[20]
Yi Xu, Junjie Ou, Hui Xu, and Luoyi Fu. 2023. Temporal knowledge graph reasoning with historical contrastive learning. In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7--14, 2023. Brian Williams, Yiling Chen, and Jennifer Neville, (Eds.) AAAI Press, 4765--4773. https://ojs.aaai.org/index.php/AAAI/article/view/25601.
[21]
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings. Yoshua Bengio and Yann LeCun, (Eds.) http://arxiv.org/abs/1412.6575.
[22]
Emanuele Zappala, Antonio Henrique de Oliveira Fonseca, Josue Ortega Caro, and David van Dijk. 2022. Neural integral equations. CoRR, abs/2209.15190. arXiv: 2209.15190.
[23]
Mengqi Zhang, Yuwei Xia, Qiang Liu, ShuWu, and LiangWang. 2023. Learning long- and short-term representations for temporal knowledge graph reasoning. In Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX, USA, 30 April 2023 - 4 May 2023. Ying Ding, Jie Tang, Juan F. Sequeda, Lora Aroyo, Carlos Castillo, and Geert-Jan Houben, (Eds.) ACM, 2412--2422.
Index Terms
- Attentional Neural Integral Equation for Temporal Knowledge Graph Forecasting
Recommendations
Temporal inductive path neural network for temporal knowledge graph reasoning
AbstractTemporal Knowledge Graph (TKG) is an extension of traditional Knowledge Graph (KG) that incorporates the dimension of time. Reasoning on TKGs is a crucial task that aims to predict future facts based on historical occurrences. The key challenge ...
Fractional oscillator equation - Transformation into integral equation and numerical solution
In this paper we propose a numerical solution of a fractional oscillator equation (being a class of the fractional Euler-Lagrange equation). At first, we convert the fractional differential equation of order α 0 to an equivalent integral equation (...
Multi-hop temporal knowledge graph reasoning with temporal path rules guidance
AbstractTemporal Knowledge graph (TKG) is a key component of artificial intelligence. In recent years, many large-scale temporal knowledge graphs have been produced and put into practical applications. At present, researchers have proposed many methods ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2024
5705 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 October 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
Conference
CIKM '24
Sponsor:
CIKM '24: The 33rd ACM International Conference on Information and Knowledge Management
October 21 - 25, 2024
ID, Boise, USA
Acceptance Rates
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 82Total Downloads
- Downloads (Last 12 months)82
- Downloads (Last 6 weeks)53
Reflects downloads up to 12 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in