More Web Proxy on the site http://driver.im/

research-article

Improving Graph Neural Networks with Structural Adaptive Receptive Fields

Authors:

Guojie SongAuthors Info & Claims

WWW '21: Proceedings of the Web Conference 2021

Pages 2438 - 2447

https://doi.org/10.1145/3442381.3449896

Published: 03 June 2021 Publication History

Abstract

The abundant information in graphs helps us to learn more expressive node representations. Different nodes in the neighborhood have different importance to the central node. Thus, average weight aggregation in most Graph Neural Networks would fail to model such difference. GAT-based models introduce the attention mechanism to solve this problem, but they ignore the rich structural information and may suffer from the problem of over-smoothing. In this paper, we propose Graph Neural Networks with STructural Adaptive Receptive fields (STAR-GNN), which adaptively construct a receptive field for each node with structural information and further achieve better aggregation of information. Firstly, we model local structural distribution based on anonymous random walks, followed by using the structural information to construct receptive fields guided with mutual information. Then, as the generated receptive fields are irregular, we design a sub-graph aggregator to boost node representations and theoretically prove that it has the ability to capture the complex structures in receptive fields. Experimental results demonstrate the power of STAR-GNN in learning structural receptive fields adaptively and encoding more informative structural characteristics in real-world networks.

References

[1]

Ittai Abraham, Yair Bartal, and Ofer Neimany. 2006. Advances in metric embedding theory. In Proceedings of the thirty-eighth annual ACM symposium on Theory of computing. 271–286.

Digital Library

[2]

Luís B Almeida. 2003. MISEP–Linear and nonlinear ICA based on mutual information. Journal of Machine Learning Research 4, Dec (2003), 1297–1318.

[3]

Anonymous. 2021. Learning Discrete Adaptive Receptive Fields for Graph Convolutional Networks. In Submitted to International Conference on Learning Representations. https://openreview.net/forum?id=pHkBwAaZ3UK under review.

[4]

Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. 2018. Mutual information neural estimation. In International Conference on Machine Learning. 531–540.

[5]

Anthony J Bell and Terrence J Sejnowski. 1995. An information-maximization approach to blind separation and blind deconvolution. Neural computation 7, 6 (1995), 1129–1159.

Digital Library

[6]

Benjamin Bloem-Reddy and Yee Whye Teh. 2019. Probabilistic symmetry and invariant neural networks. arXiv preprint arXiv:1901.06082(2019).

[7]

Emmanuel J Candès and Benjamin Recht. 2009. Exact matrix completion via convex optimization. Foundations of Computational mathematics 9, 6 (2009), 717.

[8]

Zhengdao Chen, Soledad Villar, Lei Chen, and Joan Bruna. 2019. On the equivalence between graph isomorphism testing and function approximation with gnns. In Advances in Neural Information Processing Systems. 15894–15902.

[9]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable Feature Learning for Networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2016).

Digital Library

[10]

Aditya Grover, Aaron Zweig, and Stefano Ermon. 2019. Graphite: Iterative Generative Modeling of Graphs. In ICML(Proceedings of Machine Learning Research, Vol. 97). PMLR, 2434–2444.

[11]

William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017).

Digital Library

[12]

Arman Hasanzadeh, Ehsan Hajiramezanali, Krishna R. Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. 2019. Semi-Implicit Graph Variational Auto-Encoders. In NeurIPS. 10711–10722.

[13]

Yifan Hou, Jian Zhang, James Cheng, Kaili Ma, Richard T. B. Ma, Hongzhi Chen, and Ming-Chang Yang. 2020. Measuring and Improving the Use of Graph Information in Graph Neural Networks. In ICLR. OpenReview.net.

[14]

Aapo Hyvärinen and Erkki Oja. 2000. Independent component analysis: algorithms and applications. Neural networks 13, 4-5 (2000), 411–430.

[15]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR 2017).

[16]

Risi Kondor and Shubhendu Trivedi. 2018. On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups. In ICML(Proceedings of Machine Learning Research, Vol. 80). PMLR, 2752–2760.

[17]

Lingpeng Kong, Cyprien de Masson d’Autume, Wang Ling, Lei Yu, Zihang Dai, and Dani Yogatama. 2019. A Mutual Information Maximization Perspective of Language Representation Learning. arXiv preprint arXiv:1910.08350(2019).

[18]

Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. 2016. Gated Graph Sequence Neural Networks. In Proceedings of the 4th International Conference on Learning Representations (ICLR 2016).

[19]

Ziqi Liu, Chaochao Chen, Longfei Li, Jun Zhou, Xiaolong Li, Le Song, and Yuan Qi. 2019. GeniePath: Graph Neural Networks with Adaptive Receptive Paths. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI 2019).

Digital Library

[20]

Silvio Micali and Zeyuan Allen Zhu. 2016. Reconstructing markov processes from independent and anonymous experiments. Discrete Applied Mathematics 200 (2016), 108–122.

Digital Library

[21]

Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. 2016. f-gan: Training generative neural samplers using variational divergence minimization. In Advances in neural information processing systems. 271–279.

[22]

Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang. 2020. Graph Representation Learning via Graphical Mutual Information Maximization. In Proceedings of The Web Conference 2020. 259–270.

Digital Library

[23]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2014).

Digital Library

[24]

Benjamin Recht, Maryam Fazel, and Pablo A Parrilo. 2010. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM review 52, 3 (2010), 471–501.

[25]

Renato Renner and Ueli Maurer. 2002. About the mutual (conditional) information. In Proc. IEEE ISIT. 364.

[26]

Balasubramaniam Srinivasan and Bruno Ribeiro. 2020. On the Equivalence between Positional Node Embeddings and Structural Graph Representations. In ICLR. OpenReview.net.

[27]

Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. 2020. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In ICLR. OpenReview.net.

[28]

Da Tang, Dawen Liang, Tony Jebara, and Nicholas Ruozzi. 2019. Correlated Variational Auto-Encoders. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Vol. 97. PMLR, 6135–6144. http://proceedings.mlr.press/v97/tang19b.html

[29]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proceedings of the 24th international conference on world wide web. 1067–1077.

Digital Library

[30]

Michael Tschannen, Josip Djolonga, Paul K Rubenstein, Sylvain Gelly, and Mario Lucic. 2019. On mutual information maximization for representation learning. arXiv preprint arXiv:1907.13625(2019).

[31]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations (ICLR 2018).

[32]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum?id=ryGs6iA5Km

[33]

Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation Learning on Graphs with Jumping Knowledge Networks. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, July 10-15, 2018(Proceedings of Machine Learning Research, Vol. 80), Jennifer G. Dy and Andreas Krause (Eds.). PMLR, 5449–5458.

[34]

Jiaxuan You, Rex Ying, and Jure Leskovec. 2019. Position-aware Graph Neural Networks. In ICML(Proceedings of Machine Learning Research, Vol. 97). PMLR, 7134–7143.

[35]

Kai Zhang, Yaokang Zhu, Jun Wang, and Jie Zhang. 2020. Adaptive Structural Fingerprints for Graph Attention Networks. In ICLR. OpenReview.net.

Cited By

Wang KZhang GZhang XFang JWu XLi GPan SHuang WLiang YBaeza-Yates RBonchi F(2024)The Heterophilic Snowflake Hypothesis: Training and Empowering GNNs for Heterophilic GraphsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671791(3164-3175)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671791
Jiang BZhang ZGe SWang BWang XTang J(2024)Learning Graph Attentions via Replicator DynamicsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.339330046:12(7720-7727)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3393300
Li ZWang JChen ZWu KWei YHuang H(2024)Adaptive Graph Neural Networks for Cold-Start Multimedia Recommendation2024 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM59182.2024.00027(201-210)Online publication date: 9-Dec-2024
https://doi.org/10.1109/ICDM59182.2024.00027
Show More Cited By

Improving Graph Neural Networks with Structural Adaptive Receptive Fields
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Improved Izhikevich neurons for spiking neural networks

Spiking neural networks constitute a modern neural network paradigm that overlaps machine learning and computational neurosciences. Spiking neural networks use neuron models that possess a great degree of biological realism. The most realistic model of ...
Dynamics of cortical columns – self-organization of receptive fields
ICANN'05: Proceedings of the 15th international conference on Artificial Neural Networks: biological Inspirations - Volume Part I

We present a system of differential equations which abstractly models neural dynamics and synaptic plasticity of a cortical macrocolumn. The equations assume inhibitory coupling between minicolumn activities and Hebbian type synaptic plasticity of ...
Neural networks with local receptive fields and superlinear VC Dimension

Local receptive field neurons comprise such well-known and widely used unit types as radial basis function (RBF) neurons and neurons with center-surround receptive field. We study the Vapnik-Chervonenkis (VC) dimension of feedforward neural networks ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '21: Proceedings of the Web Conference 2021

April 2021

4054 pages

ISBN:9781450383127

DOI:10.1145/3442381

Editors:
Jure Leskovec
Stanford
,
Marko Grobelnik
Jožef Stefan Institute
,
Marc Najork
Google
,
Jie Tang
Tsinghua University
,
Leila Zia
Wikimedia Foundation

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '21

Sponsor:

SIGWEB

WWW '21: The Web Conference 2021

April 19 - 23, 2021

Ljubljana, Slovenia

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
633
Total Downloads

Downloads (Last 12 months)42
Downloads (Last 6 weeks)4

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang KZhang GZhang XFang JWu XLi GPan SHuang WLiang YBaeza-Yates RBonchi F(2024)The Heterophilic Snowflake Hypothesis: Training and Empowering GNNs for Heterophilic GraphsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671791(3164-3175)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671791
Jiang BZhang ZGe SWang BWang XTang J(2024)Learning Graph Attentions via Replicator DynamicsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.339330046:12(7720-7727)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3393300
Li ZWang JChen ZWu KWei YHuang H(2024)Adaptive Graph Neural Networks for Cold-Start Multimedia Recommendation2024 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM59182.2024.00027(201-210)Online publication date: 9-Dec-2024
https://doi.org/10.1109/ICDM59182.2024.00027
Ip PZhang SWei XChan TU L(2024)HFGNN: Efficient Graph Neural Networks Using Hub-Fringe Structures2024 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM59182.2024.00022(151-160)Online publication date: 9-Dec-2024
https://doi.org/10.1109/ICDM59182.2024.00022
Zhang WGao XYang LCao MHuang PShan JYin HCui B(2024)BIM: Improving Graph Neural Networks with Balanced Influence Maximization2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00228(2931-2944)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00228
Haddadian PBooryaee RAbedian RMoeini A(2024)Multi-hop Attention-based Graph Pooling: A Personalized PageRank Perspective2024 Third International Conference on Distributed Computing and High Performance Computing (DCHPC)10.1109/DCHPC60845.2024.10454077(1-7)Online publication date: 14-May-2024
https://doi.org/10.1109/DCHPC60845.2024.10454077
Xue GZhong MQian TLi J(2024)PSA-GNN: An augmented GNN framework with priori subgraph knowledgeNeural Networks10.1016/j.neunet.2024.106155173(106155)Online publication date: May-2024
https://doi.org/10.1016/j.neunet.2024.106155
Guo GWang CYan BLou YFeng HZhu JChen JHe FYu P(2023)Learning Adaptive Node Embeddings Across GraphsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.316021135:6(6028-6042)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TKDE.2022.3160211
Xing C(2023)Recommendation Model of Graph Neural Network Based on Neighbor Aware Interaction2023 2nd Asia Conference on Electrical, Power and Computer Engineering (EPCE)10.1109/EPCE58798.2023.00038(178-184)Online publication date: Apr-2023
https://doi.org/10.1109/EPCE58798.2023.00038
Fu QDu LMao HChen XFang WHan SZhang DKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Neuron with steady response leads to better generalizationProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3601800(21046-21058)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3601800
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten