Abstract
Network embedding aims to learn low-dimensional representations of nodes while capturing structure information of networks. It has achieved great success on many tasks of network analysis such as link prediction and node classification. Most of existing network embedding algorithms focus on how to learn static homogeneous networks effectively. However, networks in the real world are more complex, e.g.., networks may consist of several types of nodes and edges (called heterogeneous information) and may vary over time in terms of dynamic nodes and edges (called evolutionary patterns). Limited work has been done for network embedding of dynamic heterogeneous networks as it is challenging to learn both evolutionary and heterogeneous information simultaneously. In this paper, we propose a novel dynamic heterogeneous network embedding method, termed as DyHATR, which uses hierarchical attention to learn heterogeneous information and incorporates recurrent neural networks with temporal attention to capture evolutionary patterns. We benchmark our method on four real-world datasets for the task of link prediction. Experimental results show that DyHATR significantly outperforms several state-of-the-art baselines.
L. Yang—Equal Contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bian, R., Koh, Y.S., Dobbie, G., Divoli, A.: Network embedding and change modeling in dynamic heterogeneous networks. In: SIGIR, pp. 861–864 (2019)
Cai, H., Zheng, V.W., Chang, K.: A comprehensive survey of graph embedding: problems, techniques, and applications. IEEE TKDE 30(09), 1616–1637 (2018)
Cen, Y., Zou, X., Zhang, J., Yang, H., Zhou, J., Tang, J.: representation learning for attributed multiplex heterogeneous network. In: SIGKDD, pp. 1358–68 (2019)
Chen, J., et al.: E-LSTM-D: a deep learning framework for dynamic network link prediction. IEEE Trans. Syst. Man Cybern. Syst. 1–14 (2019)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:abs/1406.1078 (2014)
Cui, P., Wang, X., Pei, J., Zhu, W.: A survey on network embedding. IEEE TKDE 31(05), 833–852 (2019)
Dong, Y., Chawla, N.V., Swami, A.: Metapath2Vec: scalable Representation Learning for Heterogeneous Networks. In: SIGKDD, pp. 135–144 (2017)
Du, L., Wang, Y., Song, G., Lu, Z., Wang, J.: Dynamic network embedding : an extended approach for skip-gram based network embedding. In: IJCAI, pp. 2086–2092 (2018)
Fang, H., Wu, F., Zhao, Z., Duan, X., Zhuang, Y., Ester, M.: Community-based question answering via heterogeneous social network learning. In: AAAI, pp. 122–128 (2016)
Gligorijević, V., Barot, M., Bonneau, R.: deepNF: deep network fusion for protein function prediction. Bioinformatics 34(22), 3873–3881 (2018)
Goyal, P., Chhetri, S.R., Canedo, A.: dyngraph2vec: capturing network dynamics using dynamic graph representation learning. Knowl. Based Syst. 187, 104816 (2019)
Goyal, P., Kamra, N., He, X., Liu, Y.: Dyngem: Deep embedding method for dynamic graphs. arXiv preprint arXiv:1805.11273 (2018)
Grover, A., Leskovec, J.: Node2Vec: scalable feature learning for networks. In: SIGKDD, pp. 855–864 (2016)
Hamilton, W.L., Ying, R., Leskovec, J.: Inductive representation learning on large graphs. In: NIPS, pp. 1024–1034 (2017)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)
Kong, C., Li, H., Zhang, L., Zhu, H., Liu, T.: Link prediction on dynamic heterogeneous information networks. In: CSoNet (2019)
Leskovec, J., Krevl, A.: SNAP Datasets: Stanford large network dataset collection, June 2014. http://snap.stanford.edu/data
Li, T., Zhang, J., Yu, P.S., Zhang, Y., Yan, Y.: Deep dynamic network embedding for link prediction. IEEE Access 6, 29219–29230 (2018)
Lu, Y., Wang, X., Shi, C., Yu, P.S., Ye, Y.: Temporal network embedding with micro- and macro-dynamics. In: CIKM, p. 469–478 (2019)
Milani Fard, A., Bagheri, E., Wang, K.: Relationship prediction in dynamic heterogeneous information networks. In: Azzopardi, L., Stein, B., Fuhr, N., Mayr, P., Hauff, C., Hiemstra, D. (eds.) ECIR 2019. LNCS, vol. 11437, pp. 19–34. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-15712-8_2
Nelson, W., Zitnik, M., Wang, B., Leskovec, J., Goldenberg, A., Sharan, R.: To embed or not: network embedding as a paradigm in computational biology. Front. Genet. 10, 381 (2019)
Pareja, A., et al.: EvolveGCN: evolving graph convolutional networks for dynamic graphs. In: AAAI (2020)
Peng, J., Xue, H., Wei, Z., Tuncali, I., Hao, J., Shang, X.: Integrating multi-network topology for gene function prediction using deep neural networks. Brief. Bioinform. (2020)
Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: SIGKDD, pp. 701–710 (2014)
Sajadmanesh, S., Bazargani, S., Zhang, J., Rabiee, H.R.: Continuous-time relationship prediction in dynamic heterogeneous information networks. ACM TKDD 13(4), 44:1–44:31 (2019)
Sankar, A., Wu, Y., Gou, L., Zhang, W., Yang, H.: Dynamic graph representation learning via self-attention networks. In: Workshop on Representation Learning on Graphs and Manifolds in ICLR (2019)
Shi, C., Hu, B., Zhao, W.X., Yu, P.S.: Heterogeneous information network embedding for recommendation. IEEE TKDE 31(2), 357–370 (2019)
Singer, U., Guy, I., Radinsky, K.: Node embedding over temporal graphs. In: IJCAI, pp. 4605–4612 (2019)
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: LINE: large-scale information network embedding. In: WWW, pp. 1067–1077 (2015)
Trivedi, R., Farajtabar, M., Biswal, P., Zha, H.: DyRep: learning representations over dynamic graphs. In: ICLR (2019)
Tu, C., Liu, H., Liu, Z., Sun, M.: CANE: context-aware network embedding for relation modeling. In: ACL, pp. 1722–1731, July 2017
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. ICLR (2018)
Wang, X., et al.: Heterogeneous graph attention network. In: WWW, pp. 2022–2032 (2019)
Wen, Y., Guo, L., Chen, Z., Ma, J.: Network embedding based recommendation method in social networks. In: WWW, pp. 11–12 (2018)
Xu, D., Cheng, W., Luo, D., Liu, X., Zhang, X.: Spatio-temporal attentive RNN for node classification in temporal attributed graphs. In: IJCAI (2019)
Xue, H., Peng, J., Li, J., Shang, X.: Integrating multi-network topology via deep semi-supervised node embedding. In: CIKM, pp. 2117–2120 (2019)
Yang, L., Xiao, Z., Jiang, W., Wei, Y., Hu, Y., Wang, H.: Dynamic heterogeneous graph embedding using hierarchical attentions. In: Jose, J.M., et al. (eds.) ECIR 2020. LNCS, vol. 12036, pp. 425–432. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45442-5_53
Yin, Y., Ji, L., Zhang, J., Pei, Y.: DHNE: network representation learning method for dynamic heterogeneous networks. IEEE Access 7, 134782–134792 (2019)
Zhang, C., Song, D., Huang, C., Swami, A., Chawla, N.V.: Heterogeneous graph neural network. In: SIGKDD, pp. 793–803 (2019)
Zhang, H., Qiu, L., Yi, L., Song, Y.: Scalable multiplex network embedding. In: IJCAI, pp. 3082–3088 (2018)
Zhang, Z., Cui, P., Pei, J., Wang, X., Zhu, W.: Timers: error-bounded SVD restart on dynamic networks. In: AAAI (2018)
Zhou, L.k., Yang, Y., Ren, X., Wu, F., Zhuang, Y.: Dynamic network embedding by modeling triadic closure process. In: AAAI (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xue, H., Yang, L., Jiang, W., Wei, Y., Hu, Y., Lin, Y. (2021). Modeling Dynamic Heterogeneous Network for Link Prediction Using Hierarchical Attention with Temporal RNN. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2020. Lecture Notes in Computer Science(), vol 12457. Springer, Cham. https://doi.org/10.1007/978-3-030-67658-2_17
Download citation
DOI: https://doi.org/10.1007/978-3-030-67658-2_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67657-5
Online ISBN: 978-3-030-67658-2
eBook Packages: Computer ScienceComputer Science (R0)