Just Accepted
Heterogeneous Graph Neural Networks using Self-supervised Reciprocally Contrastive Learning
Authors: 
Cuiying Huo, Dongxiao He, Yawen Li, Di Jin, Jianwu Dang, Witold Pedrycz, Lingfei Wu, Weixiong ZhangAuthors Info & Claims
Accepted on 08 November 2024
Abstract
Heterogeneous graph neural network (HGNN) is a popular technique for modeling and analyzing heterogeneous graphs. Most existing HGNN-based approaches are supervised or semi-supervised learning methods requiring graphs to be annotated, which is costly and time-consuming. Self-supervised contrastive learning has been proposed to address the problem of requiring annotated data by mining intrinsic properties in the given data. However, the existing contrastive learning methods are not suitable for heterogeneous graphs because they construct contrastive views only based on data perturbation or pre-defined structural properties (e.g., meta-path) in graph data while ignoring noises in node attributes and graph topologies. We develop a robust heterogeneous graph contrastive learning approach, namely HGCL, which introduces two views on respective guidances of node attributes and graph topologies and integrates and enhances them by a reciprocally contrastive mechanism to better model heterogeneous graphs. In this new approach, we adopt distinct but suitable attribute and topology fusion mechanisms in the two views, which are conducive to mining relevant information in attributes and topologies separately. We further use both attribute similarity and topological correlation to construct high-quality contrastive samples. Extensive experiments on four large real-world heterogeneous graphs demonstrate the superiority and robustness of HGCL over several state-of-the-art methods.
References
[1]
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. 2020. A Simple Framework for Contrastive Learning of Visual Representations. In ICML, Vol. 119. 1597–1607.
[2]
Yu Chen, Lingfei Wu, and Mohammed J. Zaki. 2020. Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings. In NeurIPS. 19314–19326.
[3]
Nurendra Choudhary, Charu C. Aggarwal, Karthik Subbian, and Chandan K. Reddy. 2022. Self-supervised Short-text Modeling through Auxiliary Context Generation. ACM Trans. Intell. Syst. Technol. 13, 3 (2022), 51:1–51:21.
[4]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL. 4171–4186.
[5]
Yuxiao Dong, Nitesh V. Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable Representation Learning for Heterogeneous Networks. In SIGKDD. 135–144.
[6]
Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. 2019. Metapath-guided Heterogeneous Graph Neural Network for Intent Recommendation. In SIGKDD. 2478–2486.
[7]
Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding. In WWW. 2331–2341.
[8]
William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NeurIPS. 1024–1034.
[9]
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. 2020. Momentum Contrast for Unsupervised Visual Representation Learning. In CVPR. 9726–9735.
[10]
Binbin Hu, Yuan Fang, and Chuan Shi. 2019. Adversarial Learning on Heterogeneous Information Networks. In SIGKDD. 120–129.
[11]
Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020. Heterogeneous Graph Transformer. In WWW. 2704–2710.
[12]
Xunqiang Jiang, Tianrui Jia, Yuan Fang, Chuan Shi, Zhe Lin, and Hui Wang. 2021. Pre-training on Large-Scale Heterogeneous Graph. In SIGKDD. 756–766.
[13]
Di Jin, Cuiying Huo, Chundong Liang, and Liang Yang. 2021. Heterogeneous Graph Neural Network via Attribute Completion. In WWW. 391–400.
[14]
Ming Jin, Yizhen Zheng, Yuan-Fang Li, Chen Gong, Chuan Zhou, and Shirui Pan. 2021. Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning. In IJCAI. 1477–1483.
[15]
Thomas N Kipf and Max Welling. 2016. Variational Graph Auto-Encoders. NeurIPS Workshop on Bayesian Deep Learning (2016), 1–3.
[16]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
[17]
Johannes Klicpera, Stefan Weißenberger, and Stephan Günnemann. 2019. Diffusion Improves Graph Learning. In NeurIPS. 13333–13345.
[18]
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In ICLR.
[19]
Xueqi Li, Guoqing Xiao, Yuedan Chen, Kenli Li, and Gao Cong. 2024. Accurate and Scalable Graph Convolutional Networks for Recommendation Based on Subgraph Propagation. IEEE Transactions on Knowledge and Data Engineering (2024), 1–12.
[20]
Yixin Liu, Shirui Pan, Ming Jin, Chuan Zhou, Feng Xia, and Philip S. Yu. 2021. Graph Self-Supervised Learning: A Survey. arXiv preprint arXiv:2103.00111 (2021).
[21]
Yuanfu Lu, Chuan Shi, Linmei Hu, and Zhiyuan Liu. 2019. Relation Structure-Aware Heterogeneous Information Network Embedding. In AAAI. 4456–4463.
[22]
Qingsong Lv, Ming Ding, Qiang Liu, Yuxiang Chen, Wenzheng Feng, Siming He, Chang Zhou, Jianguo Jiang, Yuxiao Dong, and Jie Tang. 2021. Are we really making much progress?: Revisiting, benchmarking and refining heterogeneous graph neural networks. In SIGKDD. 1150–1160.
[23]
Chanyoung Park, Jiawei Han, and Hwanjo Yu. 2020. Deep multiplex graph infomax: Attentive multiplex network embedding using global information. Knowledge-Based Systems 197 (2020), 105861.
[24]
Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang. 2020. Graph representation learning via graphical mutual information maximization. In WWW. 259–270.
[25]
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020. GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training. In SIGKDD. 1150–1160.
[26]
Wei Shen, Jiawei Han, Jianyong Wang, Xiaojie Yuan, and Zhenglu Yang. 2018. SHINE+: A General Framework for Domain-Specific Entity Linking with Heterogeneous Information Networks. IEEE Trans. Knowl. Data Eng. 30, 2 (2018), 353–366.
[27]
Maxim Shevtsov, Alexei Soupikov, and Alexander Kapustin. 2007. Highly Parallel Fast KD-tree Construction for Interactive Ray Tracing of Dynamic Scenes. Comput. Graph. Forum 26, 3 (2007), 395–404.
[28]
Chuan Shi, Yitong Li, Jiawei Zhang, Yizhou Sun, and Philip S. Yu. 2017. A Survey of Heterogeneous Information Network Analysis. IEEE Trans. Knowl. Data Eng. 29, 1 (2017), 17–37.
[29]
Yizhou Sun and Jiawei Han. 2013. Mining heterogeneous information networks: a structural analysis approach. ACM SIGKDD Explorations Newsletter 14, 2 (2013), 20–28.
[30]
Johan A. K. Suykens. 2001. Support Vector Machines: A Nonlinear Modelling and Control Perspective. European Journal of Control 7, 2-3 (2001), 311–327.
[31]
Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Mehdi Azabou, Eva L. Dyer, Rémi Munos, Petar Velickovic, and Michal Valko. 2022. Large-Scale Representation Learning on Graphs via Bootstrapping. In ICLR.
[32]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.
[33]
Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep Graph Infomax. In ICLR.
[34]
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous Graph Attention Network. In WWW. 2022–2032.
[35]
Xiao Wang, Nian Liu, Hui Han, and Chuan Shi. 2021. Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning. In SIGKDD. 1726–1736.
[36]
Zhen Wang, Chunjiang Mu, Shuyue Hu, Chen Chu, and Xuelong Li. 2022. Modelling the Dynamics of Regret Minimization in Large Agent Populations: a Master Equation Approach. In IJCAI. 534–540.
[37]
Lingfei Wu, Yu Chen, Kai Shen, Xiaojie Guo, Hanning Gao, Shucheng Li, Jian Pei, and Bo Long. 2021. Graph neural networks for natural language processing: A survey. arXiv preprint arXiv:2106.06090 (2021).
[38]
Lingfei Wu, Peng Cui, Jian Pei, and Liang Zhao. 2022. Graph Neural Networks: Foundations, Frontiers, and Applications. Springer Singapore, Singapore. 725 pages.
[39]
Yaochen Xie, Zhao Xu, Zhengyang Wang, and Shuiwang Ji. 2021. Self-Supervised Learning of Graph Neural Networks: A Unified Review. arXiv preprint arXiv:2102.10757 (2021).
[40]
Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu and L. Wang. 2020. Graph Contrastive Learning with Adaptive Augmentation. In WWW. 2069–2080.
[41]
Carl Yang, Yuxin Xiao, Yu Zhang, Yizhou Sun, and Jiawei Han. 2020. Heterogeneous Network Representation Learning: A Unified Framework with Survey and Benchmark. IEEE Trans. Knowl. Data Eng. 34, 10 (2020), 4854–4873.
[42]
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph Contrastive Learning with Augmentations. In NeurIPS. 5812–5823.
[43]
Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J. Kim. 2019. Graph Transformer Networks. In NeurIPS. 11960–11970.
[44]
Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V. Chawla. 2019. Heterogeneous Graph Neural Network. In SIGKDD. 793–803.
[45]
Jianan Zhao, Xiao Wang, Chuan Shi, Binbin Hu, Guojie Song, and Yanfang Ye. 2021. Heterogeneous Graph Structure Learning for Graph Neural Networks. In AAAI. 4697–4705.
[46]
Jianan Zhao, Xiao Wang, Chuan Shi, Zekuan Liu, and Yanfang Ye. 2020. Network Schema Preserving Heterogeneous Information Network Embedding. In IJCAI. 1366–1372.
[47]
Yu Zhou, Haixia Zheng, Xin Huang, Shufeng Hao, Dengao Li, and Jumin Zhao. 2022. Graph Neural Networks: Taxonomy, Advances, and Trends. ACM Trans. Intell. Syst. Technol. 13, 1 (2022), 15:1–15:54.
[48]
Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang. 2021. Deep Graph Structure Learning for Robust Representations: A Survey. arXiv preprint arXiv:2103.03036 (2021).
[49]
Yanqiao Zhu, Yichen Xu, Hejie Cui, Carl Yang, Qiang Liu, and Shu Wu. 2021. Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning. arXiv preprint arXiv:2108.13886 (2021).
[50]
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2020. Deep Graph Contrastive Representation Learning. arXiv preprint arXiv:2006.04131 (2020).
Index Terms
- Heterogeneous Graph Neural Networks using Self-supervised Reciprocally Contrastive Learning
Recommendations
Training Heterogeneous Graph Neural Networks using Bandit Sampling
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementGraph neural networks (GNNs) have gained significant attention across diverse areas due to their superior performance in learning graph representations. While GNNs exhibit superior performance compared to other methods, they are primarily designed for ...
Self-supervised Deep Heterogeneous Graph Neural Networks with Contrastive Learning
Computational Science – ICCS 2023AbstractHeterogeneous graph neural networks have shown superior capabilities on graphs that contain multiple types of entities with rich semantic information. However, they are usually (semi-)supervised learning methods which rely on costly task-specific ...
Graph Self-supervised Learning with Augmentation-aware Contrastive Learning
WWW '23: Proceedings of the ACM Web Conference 2023Graph self-supervised learning aims to mine useful information from unlabeled graph data, and has been successfully applied to pre-train graph representations. Many existing approaches use contrastive learning to learn powerful embeddings by learning ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
EISSN:2157-6912
Table of ContentsCopyright © 2024 Copyright held by the owner/author(s).
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Online AM: 05 December 2024
Accepted: 08 November 2024
Revised: 28 June 2024
Received: 24 February 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 75Total Downloads
- Downloads (Last 12 months)75
- Downloads (Last 6 weeks)75
Reflects downloads up to 11 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in