[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3459637.3482391acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Semi-supervised Multi-label Learning for Graph-structured Data

Published: 30 October 2021 Publication History

Abstract

The semi-supervised multi-label classification problem primarily deals with Euclidean data, such as text with a 1D grid of tokens and images with a 2D grid of pixels. However, the non-Euclidean graph-structured data naturally and constantly appears in semi-supervised multi-label learning tasks from various domains like social networks, citation networks, and protein-protein interaction (PPI) networks. Moreover, the existing popular node embedding methods, like Graph Neural Networks (GNN), focus on graphs with simplex labels and tend to neglect label correlations in the multi-label setting, so the easy adaption proves empirically ineffective. Therefore, graph representation learning for the semi-supervised multi-label learning task is crucial and challenging. In this work, we incorporate the idea of label embedding into our proposed model to capture both network topology and higher-order multi-label correlations. The label embedding is generated along with the node embedding based on the topological structure to serve as the prototype center for each class. Moreover, the similarity of the label embedding and node embedding can be used as a confidence vector to guide the label smoothing process, formulating as a margin ranking optimization problem to learn the second-order relations between labels. Extensive experiments on real-world datasets from various domains demonstrate that our model significantly outperforms the state-of-the-art models for node-level tasks.

Supplementary Material

MP4 File (CIKM21-rgfp0958.mp4)
Presentation video

References

[1]
Uchenna Akujuobi, Yufei Han, Qiannan Zhang, and Xiangliang Zhang. 2019. Collaborative Graph Walk for Semi-Supervised Multi-label Node Classification. In ICDM. IEEE, 1--10.
[2]
Matthew R. Boutell, Jiebo Luo, Xipeng Shen, and Christopher M. Brown. 2004. Learning multi-label scene classification. Pattern Recognit. 37, 9 (2004), 1757--1771.
[3]
Bobby-Joe Breitkreutz, Chris Stark, Teresa Reguly, Lorrie Boucher, Ashton Bre- itkreutz, Michael S. Livstone, Rose Oughtred, Daniel H. Lackner, Jürg Bähler, Valerie Wood, Kara Dolinski, and Mike Tyers. 2008. The BioGRID Interaction Database: 2008 update. Nucleic Acids Res. 36, Database-Issue (2008), 637--640.
[4]
Sandro Cavallari, Vincent W. Zheng, Hongyun Cai, Kevin Chen-Chuan Chang, and Erik Cambria. 2017. Learning Community Embedding with Community Detection and Node Embedding on Graphs. Association for Computing Machinery, New York, NY, USA, 377--386. https://doi.org/10.1145/3132847.3132925
[5]
Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu. 2019. A Survey on Network Embedding. IEEE Trans. Knowl. Data Eng. 31, 5 (2019), 833--852.
[6]
Kunal Dahiya, Deepak Saini, Anshul Mittal, Ankush Shaw, Kushal Dave, Akshay Soni, Himanshu Jain, Sumeet Agarwal, and Manik Varma. 2021. DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining (Virtual Event, Israel) (WSDM '21). Association for Computing Machinery, New York, NY, USA, 31--39. https://doi.org/10.1145/3437963.3441810
[7]
Janez Demsar. 2006. Statistical Comparisons of Classifiers over Multiple Data Sets. J. Mach. Learn. Res. 7 (2006), 1--30.
[8]
Jie Du and Chi-Man Vong. 2020. Robust Online Multilabel Learning Under Dynamic Changes in Data Distribution With Labels. IEEE Trans. Cybern. 50, 1 (2020), 374--385.
[9]
Kaisheng Gao, Jing Zhang, and Cangqi Zhou. 2019. Semi-supervised Graph Embedding for Multi-label Graph Node Classification. In Web Information Systems Engineering -- WISE 2019, Reynold Cheng, Nikos Mamoulis, Yizhou Sun, and Xin Huang (Eds.). Springer International Publishing, Cham, 555--567.
[10]
Aditya Grover and Jure Leskovec. 2016. Node2vec: Scalable Feature Learning for Networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD '16). Association for Computing Machinery, New York, NY, USA, 855--864. https://doi.org/10.1145/2939672.2939754
[11]
William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS'17). Curran Associates Inc., Red Hook, NY, USA, 1025--1035.
[12]
Matthew D. Hoffman, David M. Blei, Chong Wang, and John W. Paisley. 2013. Stochastic variational inference. J. Mach. Learn. Res. 14, 1 (2013), 1303--1347.
[13]
Qian Huang, Horace He, Abhay Singh, Ser-Nam Lim, and Austin R. Benson. 2021. Combining Label Propagation and Simple Models out-performs Graph Neural Networks. In ICLR. OpenReview.net.
[14]
Tony Jebara, Jun Wang, and Shih-Fu Chang. 2009. Graph Construction and b-Matching for Semi-Supervised Learning. In Proceedings of the 26th Annual International Conference on Machine Learning (Montreal, Quebec, Canada) (ICML '09). Association for Computing Machinery, New York, NY, USA, 441--448. https://doi.org/10.1145/1553374.1553432
[15]
Jyun-Yu Jiang, Zeyu Li, Chelsea J.-T. Ju, and Wei Wang. 2020. MARU: Meta-Context Aware Random Walks for Heterogeneous Network Representation Learning. Association for Computing Machinery, New York, NY, USA, 575--584. https://doi.org/10.1145/3340531.3412040
[16]
Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In ICLR.
[17]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR (Poster). OpenReview.net.
[18]
M. Mahoney. 2011. Large text compression benchmark. http://www. mattmahoney.net/dc/textdata
[19]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In NeurIPS. 8024--8035.
[20]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online Learning of Social Representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (New York, New York, USA) (KDD '14). Association for Computing Machinery, New York, NY, USA, 701--710. https://doi.org/10.1145/2623330.2623732
[21]
José Ramón Quevedo, Oscar Luaces, and Antonio Bahamonde. 2012. Multilabel classifiers with a probabilistic thresholding strategy. Pattern Recognit. 45, 2 (2012), 876--883.
[22]
Niloofar Rastin, Mansoor Zolghadri Jahromi, and Mohammad Taheri. 2021. A generalized weighted distance k-Nearest Neighbor for multi-label problems. Pattern Recognit. 114 (2021), 107526.
[23]
Jesse Read, Bernhard Pfahringer, Geoffrey Holmes, and Eibe Frank. 2009. Clas- sifier Chains for Multi-label Classification. In ECML/PKDD (2) (Lecture Notes in Computer Science, Vol. 5782). Springer, 254--269.
[24]
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. In Proceedings of the 31st International Conference on International Conference on Ma- chine Learning - Volume 32 (Beijing, China) (ICML'14). JMLR.org, II-1278-II-1286.
[25]
Andrea Rossi, Denilson Barbosa, Donatella Firmani, Antonio Matinata, and Paolo Merialdo. 2021. Knowledge Graph Embedding for Link Prediction: A Comparative Analysis. ACM Trans. Knowl. Discov. Data 15, 2 (2021), 14:1--14:49.
[26]
Xiao Shen, Quanyu Dai, Sitong Mao, Fu-Lai Chung, and Kup-Sze Choi. 2021. Network Together: Node Classification via Cross-Network Deep Network Em- bedding. IEEE Trans. Neural Networks Learn. Syst. 32, 5 (2021), 1935--1948.
[27]
Min Shi, Yufei Tang, and Xingquan Zhu. 2020. MLNE: Multi-Label Network Embedding. IEEE Trans. Neural Networks Learn. Syst. 31, 9 (2020), 3682--3695.
[28]
Min Shi, Yufei Tang, Xingquan Zhu, and Jianxun Liu. 2020. Multi-Label Graph Convolutional Network Representation Learning. IEEE Transactions on Big Data (2020).
[29]
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjing Wang, and Yu Sun. 2021. Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Zhi-Hua Zhou (Ed.). International Joint Conferences on Artificial Intelligence Organization, 1548--1554. https://doi.org/10.24963/ijcai.2021/214 Main Track.
[30]
Zixing Song, Xiangli Yang, Zenglin Xu, and Irwin King. 2021. Graph-based Semi- supervised Learning: A Comprehensive Review. CoRR abs/2102.13303 (2021). arXiv:2102.13303 https://arxiv.org/abs/2102.13303
[31]
Fan-Yun Sun, Meng Qu, Jordan Hoffmann, Chin-Wei Huang, and Jian Tang. 2019. vGraph: A Generative Model for Joint Community Detection and Node Representation Learning. In NeurIPS. 512--522.
[32]
Heli Sun, Fang He, Jianbin Huang, Yizhou Sun, Yang Li, Chenyu Wang, Liang He, Zhongbin Sun, and Xiaolin Jia. 2020. Network Embedding for Community Detection in Attributed Networks. ACM Trans. Knowl. Discov. Data 14, 3 (2020), 36:1--36:25.
[33]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-Scale Information Network Embedding. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW '15). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1067--1077. https://doi.org/10.1145/2736277.2741093
[34]
Lei Tang and Huan Liu. 2009. Relational Learning via Latent Social Dimensions. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining (Paris, France) (KDD '09). Association for Computing Machinery, New York, NY, USA, 817--826. https://doi.org/10.1145/1557019.1557109
[35]
Lei Tang and Huan Liu. 2009. Scalable learning of collective behavior based on sparse social dimensions. In CIKM. ACM, 1107--1116.
[36]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).
[37]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR (Poster). OpenReview.net.
[38]
Wei Wang and Min-Ling Zhang. 2020. Semi-Supervised Partial Label Learning via Confidence-Rated Margin Maximization. In NeurIPS.
[39]
Xi Wang and Gita Sukthankar. 2013. Multi-Label Relational Neighbor Classifi- cation Using Social Context Features. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Chicago, Illinois, USA) (KDD '13). Association for Computing Machinery, New York, NY, USA, 464--472. https://doi.org/10.1145/2487575.2487610
[40]
Qingyao Wu, Yunming Ye, Shen-Shyang Ho, and Shuigeng Zhou. 2014. Semisupervised multi-label collective classification ensemble for functional genomics. BMC genomics 15, 9 (2014), 1--14.
[41]
Ming-Kun Xie and Sheng-Jun Huang. 2018. Partial Multi-Label Learning. In AAAI. AAAI Press, 4302--4309.
[42]
Yuanyuan Xu, Jun Wang, Shuai An, Jinmao Wei, and Jianhua Ruan. 2018. Semi- Supervised Multi-Label Feature Selection by Preserving Feature-Label Space Consistency. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (Torino, Italy) (CIKM '18). Association for Computing Machinery, New York, NY, USA, 783--792. https://doi.org/10.1145/3269206.3271760
[43]
M. Yang, Z. Meng, and I. King. 2020. FeatureNorm: L2 Feature Normalization for Dynamic Graph Embedding. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE Computer Society, Los Alamitos, CA, USA, 731--740. https://doi.org/10.1109/ICDM50108.2020.00082
[44]
Menglin Yang, Min Zhou, Marcus Kalander, Zengfeng Huang, and Irwin King. 2021. Discrete-Time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Space. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (Virtual Event, Singapore) (KDD '21). Association for Computing Machinery, New York, NY, USA, 1975--1985. https://doi.org/10.1145/3447548.3467422
[45]
Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. 2021. A Survey on Deep Semi-supervised Learning. CoRR abs/2103.00550 (2021). arXiv:2103.00550 https://arxiv.org/abs/2103.00550
[46]
Chih-Kuan Yeh, Wei-Chieh Wu, Wei-Jen Ko, and Yu-Chiang Frank Wang. 2017. Learning Deep Latent Space for Multi-Label Classification. In AAAI. AAAI Press, 2838--2844.
[47]
Jing Zhang and Xindong Wu. 2021. Multi-Label Truth Inference for Crowdsourcing Using Mixture Models. IEEE Trans. Knowl. Data Eng. 33, 5 (2021), 2083--2095.
[48]
Min-Ling Zhang and Zhi-Hua Zhou. 2007. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognit. 40, 7 (2007), 2038--2048.
[49]
Min-Ling Zhang and Zhi-Hua Zhou. 2014. A Review on Multi-Label Learning Algorithms. IEEE Trans. Knowl. Data Eng. 26, 8 (2014), 1819--1837.
[50]
M. L. Zhang, Y. K. Li, H. Yang, and X. Y. Liu. 2020. Towards Class-Imbalance Aware Multi-Label Learning. IEEE Transactions on Cybernetics (2020), 1--13. https://doi.org/10.1109/TCYB.2020.3027509
[51]
Cangqi Zhou, Hui Chen, Jing Zhang, Qianmu Li, Dianming Hu, and Victor S. Sheng. 2021. Multi-label graph node classification with label attentive neighborhood convolution. Expert Systems with Applications 180 (2021), 115063. https://doi.org/10.1016/j.eswa.2021.115063
[52]
Xiaojin Zhu, Zoubin Ghahramani, and John Lafferty. 2003. Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions. In Proceedings of the Twentieth International Conference on International Conference on Machine Learn- ing (Washington, DC, USA) (ICML'03). AAAI Press, 912--919.
[53]
Yue Zhu, James T. Kwok, and Zhi-Hua Zhou. 2018. Multi-Label Learning with Global and Local Label Correlation. IEEE Trans. Knowl. Data Eng. 30, 6 (2018), 1081--1094.

Cited By

View all
  • (2024)Geometric View of Soft Decorrelation in Self-Supervised LearningProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671914(4338-4349)Online publication date: 25-Aug-2024
  • (2024)Multimodal Progressive Modulation Network for Micro-Video Multi-Label ClassificationIEEE Transactions on Multimedia10.1109/TMM.2024.340572426(10134-10144)Online publication date: 2024
  • (2023)Optimal block-wise asymmetric graph construction for graph-based semi-supervised learningProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669237(71135-71149)Online publication date: 10-Dec-2023
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management
October 2021
4966 pages
ISBN:9781450384469
DOI:10.1145/3459637
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 October 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. graph neural networks
  2. graph representation learning
  3. multi-label learning
  4. semi-supervised learning

Qualifiers

  • Research-article

Funding Sources

Conference

CIKM '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)109
  • Downloads (Last 6 weeks)11
Reflects downloads up to 10 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Geometric View of Soft Decorrelation in Self-Supervised LearningProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671914(4338-4349)Online publication date: 25-Aug-2024
  • (2024)Multimodal Progressive Modulation Network for Micro-Video Multi-Label ClassificationIEEE Transactions on Multimedia10.1109/TMM.2024.340572426(10134-10144)Online publication date: 2024
  • (2023)Optimal block-wise asymmetric graph construction for graph-based semi-supervised learningProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669237(71135-71149)Online publication date: 10-Dec-2023
  • (2023)Mitigating the popularity bias of graph collaborative filteringProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669074(67533-67550)Online publication date: 10-Dec-2023
  • (2023)No change, no gainProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668180(47511-47526)Online publication date: 10-Dec-2023
  • (2023)Towards Fair Financial Services for All: A Temporal GNN Approach for Individual Fairness on Transaction NetworksProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615091(2331-2341)Online publication date: 21-Oct-2023
  • (2023)Contrastive Cross-scale Graph Knowledge SynergyProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599286(3422-3433)Online publication date: 6-Aug-2023
  • (2023)Bipartite Graph Convolutional Hashing for Effective and Efficient Top-N Search in Hamming SpaceProceedings of the ACM Web Conference 202310.1145/3543507.3583219(3164-3172)Online publication date: 30-Apr-2023
  • (2023)WSFE: Wasserstein Sub-graph Feature Encoder for Effective User Segmentation in Collaborative FilteringProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592089(2521-2525)Online publication date: 19-Jul-2023
  • (2023)Lorentzian Graph Convolution Networks for Collaborative Filtering2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191482(1-8)Online publication date: 18-Jun-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media