More Web Proxy on the site http://driver.im/

research-article

Open access

Causality and Independence Enhancement for Biased Node Classification

Authors:

Xueqi ChengAuthors Info & Claims

CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Pages 203 - 212

https://doi.org/10.1145/3583780.3614804

Published: 21 October 2023 Publication History

Abstract

Most existing methods that address out-of-distribution (OOD) generalization for node classification on graphs primarily focus on a specific type of data biases, such as label selection bias or structural bias. However, anticipating the type of bias in advance is extremely challenging, and designing models solely for one specific type may not necessarily improve overall generalization performance. Moreover, limited research has focused on the impact of mixed biases, which are more prevalent and demanding in real-world scenarios. To address these limitations, we propose a novel Causality and Independence Enhancement (CIE) framework, applicable to various graph neural networks (GNNs). Our approach estimates causal and spurious features at the node representation level and mitigates the influence of spurious correlations through the backdoor adjustment. Meanwhile, independence constraint is introduced to improve the discriminability and stability of causal and spurious features in complex biased environments. Essentially, CIE eliminates different types of data biases from a unified perspective, without the need to design separate methods for each bias as before. To evaluate the performance under specific types of data biases, mixed biases, and low-resource scenarios, we conducted comprehensive experiments on five publicly available datasets. Experimental results demonstrate that our approach CIE not only significantly enhances the performance of GNNs but outperforms state-of-the-art debiased node classification methods.

References

[1]

Faruk Ahmed, Yoshua Bengio, Harm van Seijen, and Aaron Courville. 2020. Systematic generalisation with group invariant predictions. In ICLR.

[2]

Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2019. Invariant risk minimization. arXiv preprint arXiv:1907.02893 (2019).

[3]

Francis R Bach and Michael I Jordan. 2002. Kernel independent component analysis. Journal of machine learning research, Vol. 3, Jul (2002), 1--48.

Digital Library

[4]

Charles R Baker. 1973. Joint measures and cross-covariance operators. Trans. Amer. Math. Soc., Vol. 186 (1973), 273--289.

[5]

Elias Bareinboim and Judea Pearl. 2016. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, Vol. 113, 27 (2016), 7345--7352.

[6]

Martin D Buhmann. 2003. Radial basis functions: theory and implementations. Vol. 12. Cambridge university press.

Digital Library

[7]

Guoxin Chen, Yongqing Wang, Jiangli Shao, Boshen Shi, Huawei Shen, and Xueqi Cheng. 2022a. InDNI: An Infection Time Independent Method for Diffusion Network Inference. In China Conference on Information Retrieval. Springer, 63--75.

[8]

Yining Chen, Colin Wei, Ananya Kumar, and Tengyu Ma. 2020. Self-training avoids using spurious features under domain shift. NeurIPS, Vol. 33 (2020), 21061--21071.

[9]

Yongqiang Chen, Yonggang Zhang, Yatao Bian, Han Yang, MA Kaili, Binghui Xie, Tongliang Liu, Bo Han, and James Cheng. 2022c. Learning causally invariant representations for out-of-distribution generalization on graphs. NeurIPS, Vol. 35 (2022), 22131--22148.

[10]

Zhengyu Chen, Teng Xiao, and Kun Kuang. 2022b. BA-GNN: On learning bias-aware graph neural network. In 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 3012--3024.

[11]

Yoonhyuk Choi, Jiho Choi, Taewook Ko, Hyungho Byun, and Chong-Kwon Kim. 2022. Finding Heterophilic Neighbors via Confidence-based Subgraph Matching for Semi-supervised Node Classification. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 283--292.

Digital Library

[12]

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. NeurIPS, Vol. 29 (2016).

[13]

Shaohua Fan, Xiao Wang, Chuan Shi, Kun Kuang, Nian Liu, and Bai Wang. 2022. Debiased graph neural networks with agnostic label selection bias. IEEE Transactions on Neural Networks and Learning Systems (2022).

[14]

Kenji Fukumizu, Francis R Bach, and Michael I Jordan. 2004. Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces. Journal of Machine Learning Research, Vol. 5, Jan (2004), 73--99.

Digital Library

[15]

Irena Gao, Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. 2023. Out-of-Domain Robustness via Targeted Augmentations. arXiv preprint arXiv:2302.11861 (2023).

[16]

Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020. Shortcut learning in deep neural networks. Nature Machine Intelligence, Vol. 2, 11 (2020), 665--673.

[17]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 249--256.

[18]

Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. 2005. Measuring statistical dependence with Hilbert-Schmidt norms. In International conference on algorithmic learning theory. Springer, 63--77.

Digital Library

[19]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. NeurIPS, Vol. 30 (2017).

[20]

Tiantian He, Yew Soon Ong, and Lu Bai. 2021. Learning conjoint attentions for graph neural nets. NeurIPS, Vol. 34 (2021), 2641--2653.

[21]

Dan Hendrycks and Kevin Gimpel. 2016. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016).

[22]

Fereshte Khani and Percy Liang. 2021. Removing spurious features can hurt accuracy and affect groups disproportionately. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 196--205.

Digital Library

[23]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).

[24]

Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05997 (2018).

[25]

Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. Similarity of neural network representations revisited. In ICML. PMLR, 3519--3529.

[26]

Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. 2022a. Out-of-distribution generalization on graphs: A survey. arXiv preprint arXiv:2202.07987 (2022).

[27]

Haoyang Li, Ziwei Zhang, Xin Wang, and Wenwu Zhu. 2022c. Learning invariant graph representations for out-of-distribution generalization. In NeurIPS.

[28]

Zenan Li, Qitian Wu, Fan Nie, and Junchi Yan. 2022b. GraphDE: A generative framework for debiased learning and out-of-distribution detection on graphs. NeurIPS, Vol. 35 (2022), 30277--30290.

[29]

Wanyu Lin, Hao Lan, and Baochun Li. 2021. Generative causal explanations for graph neural networks. In ICML. PMLR, 6666--6679.

[30]

Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. 2020. Energy-based out-of-distribution detection. NeurIPS, Vol. 33 (2020), 21464--21475.

[31]

Vern I Paulsen and Mrinal Raghupathi. 2016. An introduction to the theory of reproducing kernel Hilbert spaces. Vol. 152. Cambridge university press.

[32]

Judea Pearl. 2009. Causal inference in statistics: An overview. Statistics surveys, Vol. 3 (2009), 96--146.

[33]

Judea Pearl. 2014. Interpretation and identification of causal mediation. Psychological methods, Vol. 19, 4 (2014), 459.

[34]

Judea Pearl et al. 2000. Models, reasoning and inference. Cambridge, UK: CambridgeUniversityPress, Vol. 19, 2 (2000).

[35]

Liang Qu, Huaisheng Zhu, Ruiqi Zheng, Yuhui Shi, and Hongzhi Yin. 2021. Imgagn: Imbalanced network embedding via generative adversarial graph networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1390--1398.

Digital Library

[36]

Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, and Peter J Liu. 2022. Out-of-Distribution Detection and Selective Generation for Conditional Language Models. arXiv preprint arXiv:2209.15558 (2022).

[37]

Elan Rosenfeld, Pradeep Ravikumar, and Andrej Risteski. 2020. The risks of invariant risk minimization. arXiv preprint arXiv:2010.05761 (2020).

[38]

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine, Vol. 29, 3 (2008), 93--93.

[39]

Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. 2021. Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624 (2021).

[40]

Le Song, Alex Smola, Arthur Gretton, Karsten M Borgwardt, and Justin Bedo. 2007. Supervised feature selection via dependence estimation. In ICML. 823--830.

[41]

Peter Spirtes. 2010. Introduction to causal inference. Journal of Machine Learning Research, Vol. 11, 5 (2010).

[42]

Yongduo Sui, Xiang Wang, Jiancan Wu, Min Lin, Xiangnan He, and Tat-Seng Chua. 2022. Causal attention for interpretable and generalizable graph classification. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1696--1705.

Digital Library

[43]

Xianfeng Tang, Huaxiu Yao, Yiwei Sun, Yiqi Wang, Jiliang Tang, Charu Aggarwal, Prasenjit Mitra, and Suhang Wang. 2020. Investigating and mitigating degree-related biases in graph convoltuional networks. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1435--1444.

Digital Library

[44]

Antonio Torralba and Alexei A Efros. 2011. Unbiased look at dataset bias. In CVPR 2011. IEEE, 1521--1528.

Digital Library

[45]

Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[46]

Julius Von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. 2021. Self-supervised learning with data augmentations provably isolates content from style. NeurIPS, Vol. 34 (2021), 16451--16467.

[47]

Yu Wang, Yuying Zhao, Neil Shah, and Tyler Derr. 2022. Imbalanced graph classification via graph-of-graph neural networks. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2067--2076.

Digital Library

[48]

Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying graph convolutional networks. In ICML. PMLR, 6861--6871.

[49]

Qitian Wu, Hengrui Zhang, Junchi Yan, and David Wipf. 2022. Handling distribution shifts on graphs: An invariance perspective. arXiv preprint arXiv:2202.02466 (2022).

[50]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, Vol. 32, 1 (2020), 4--24.

[51]

Nianzu Yang, Kaipeng Zeng, Qitian Wu, Xiaosong Jia, and Junchi Yan. 2022. Learning substructure invariance for out-of-distribution molecular representations. In NeurIPS.

[52]

Haotian Ye, Chuanlong Xie, Tianle Cai, Ruichen Li, Zhenguo Li, and Liwei Wang. 2021. Towards a theoretical framework of out-of-distribution generalization. Advances in Neural Information Processing Systems, Vol. 34 (2021), 23519--23531.

[53]

Bianca Zadrozny. 2004. Learning and evaluating classifiers under sample selection bias. In ICML. 114.

[54]

Chunting Zhou, Xuezhe Ma, Paul Michel, and Graham Neubig. 2021. Examining and combating spurious features under distribution shift. In ICML. PMLR, 12857--12867.

[55]

Jun Zhuang and Mohammad Al Hasan. 2022. Robust Node Classification on Graphs: Jointly from Bayesian Label Transition and Topology-based Label Propagation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2795--2805.

Digital Library

Cited By

Jiao LWang YLiu XLi LLiu FMa WGuo YChen PYang SHou B(2024)Causal Inference Meets Deep Learning: A Comprehensive SurveyResearch10.34133/research.04677Online publication date: 10-Sep-2024
https://doi.org/10.34133/research.0467
Chen GGuo FWang YLiu YYu PShen HCheng XSerra ESpezzano F(2024)FCS-HGNN: Flexible Multi-type Community Search in Heterogeneous Information NetworksProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679696(207-217)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679696
Wang ZXu BYuan YShen HCheng XHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive LearningProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657927(2548-2552)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657927
Show More Cited By

Index Terms

Causality and Independence Enhancement for Biased Node Classification
1. Computing methodologies
  1. Machine learning

Recommendations

Label-Consistency based Graph Neural Networks for Semi-supervised Node Classification
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Graph neural networks (GNNs) achieve remarkable success in graph-based semi-supervised node classification, leveraging the information from neighboring nodes to improve the representation learning of target node. The success of GNNs at node ...
GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks
WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

Node classification is an important research topic in graph learning. Graph neural networks (GNNs) have achieved state-of-the-art performance of node classification. However, existing GNNs address the problem where node samples for different classes are ...
Task-Adaptive Few-shot Node Classification
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Node classification is of great importance among various graph mining tasks. In practice, real-world graphs generally follow the long-tail distribution, where a large number of classes only consist of limited labeled nodes. Although Graph Neural Networks ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

October 2023

5508 pages

ISBN:9798400701245

DOI:10.1145/3583780

General Chairs:
Ingo Frommholz
University of Wolverhampton, UK
,
Frank Hopfgartner
University of Koblenz, Germany
,
Mark Lee
University of Birmingham, UK
,
Michael Oakes
University of Birmingham, UK
,
Program Chairs:
Mounia Lalmas
Spotify, UK
,
Min Zhang
Tsinghua University, China
,
Rodrygo Santos
Federal University of Minas Gerais, Brazil

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2023

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

The National Key R&D Program Young Scientists Project
The National Natural Science Foundation of China
The fellowship of China Postdoctoral Science Foundation

Conference

CIKM '23

Sponsor:

CIKM '23: The 32nd ACM International Conference on Information and Knowledge Management

October 21 - 25, 2023

Birmingham, United Kingdom

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
758
Total Downloads

Downloads (Last 12 months)610
Downloads (Last 6 weeks)53

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiao LWang YLiu XLi LLiu FMa WGuo YChen PYang SHou B(2024)Causal Inference Meets Deep Learning: A Comprehensive SurveyResearch10.34133/research.04677Online publication date: 10-Sep-2024
https://doi.org/10.34133/research.0467
Chen GGuo FWang YLiu YYu PShen HCheng XSerra ESpezzano F(2024)FCS-HGNN: Flexible Multi-type Community Search in Heterogeneous Information NetworksProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679696(207-217)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679696
Wang ZXu BYuan YShen HCheng XHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive LearningProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657927(2548-2552)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657927
Zhang YLiu SFeng YPan STian XYang J(2024)SE-FewDet: Semantic-Enhanced Feature Generation and Prediction Refinement for Few-Shot Object Detection2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650252(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650252
Jiang ZTan YChen GGuo FZhang JBai X(2024)GNN-Based Persistent K-core Community Search in Temporal Graphs2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825281(569-578)Online publication date: 15-Dec-2024
https://doi.org/10.1109/BigData62323.2024.10825281
Chen YBian YZhou KXie BHan BCheng JOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Does invariant graph learning via environment augmentation learn invariance?Proceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669252(71486-71519)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3669252

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten