More Web Proxy on the site http://driver.im/

research-article

Semi-supervised Multi-task Learning with Auxiliary data

Authors:

Ruiguang Huang,

Liangjiao LiAuthors Info & Claims

Volume 626, Issue C

Pages 626 - 639

https://doi.org/10.1016/j.ins.2023.02.091

Published: 01 May 2023 Publication History

Abstract

Compared with single-task learning, multi-tasks can obtain better classifiers by the information provided by each task. In the process of multi-task data collection, we always focus on the target task data in the training process, and ignore the non-target task data and unlabeled data that may be contained in the target task. In response to this issue, this paper introduces auxiliary or Universum into semi-supervised multi-task problem, and proposes a multi-task support vector machine (SU-MTLSVM) method based on semi-supervised learning to handle the case where each task contains the labeled, unlabeled, and Universum samples in the training set. This method introduces Universum as prior knowledge and provides high-dimensional information for semi-supervised learning, and builds a unique classifier from a large amount of unlabeled data. We then use KKT conditions and Lagrangian method to optimize the formulation of the model, and get the model parameters. Finally, we collect different data sets in the experiment part, and compare the performance of multiple baselines with the proposed method. Experiments prove that the method proposed in this paper is more effective for multi-task applications.

References

[1]

R. Yuan, Z. Li, X. Guan, L. Xu, An svm-based machine learning method for accurate internet traffic classification, Inf. Syst. Front. 12 (2010) 149–156.

[2]

Dong, G.; Pentukar, S.K. : Oclep+: one-class anomaly and intrusion detection using minimal length of emerging patterns. arXiv:1811.09842.

[3]

H. Acar, M.S. Özerdem, E. Acar, Soil moisture inversion via semiempirical and machine learning methods with full-polarization radarsat-2 and polarimetric target decomposition data: a comparative study, IEEE Access 8 (2020) 197896–197907.

[4]

M. Nunes, E.H. Gerding, F. McGroarty, M. Niranjan, A comparison of multitask and single task learning with artificial neural networks for yield curve forecasting, Expert Syst. Appl. 119 (2019) 362–375.

[5]

Aleotti, F.; Poggi, M.; Tosi, F.; Mattoccia, S. : Learning end-to-end scene flow by distilling single tasks knowledge. arXiv:1911.10090.

[6]

R. Caruana, Multitask learning: a knowledge-based source of inductive bias, in: ICML, 1993.

[7]

G. Buroni, B. Lebichot, G. Bontempi, Ast-mtl: an attention-based multi-task learning strategy for traffic forecasting, IEEE Access 9 (2021) 77359–77370.

[8]

H. Cao, S. Pu, W. Tan, J. Tong, D. Zhang, Multi-tasking u-shaped network for benign and malignant classification of breast masses, IEEE Access 8 (2020) 223396–223404.

[9]

Y. Xue, P. Beauseroy, Multi-task learning for one-class svm with additional new features, in: 2016 23rd International Conference on Pattern Recognition (ICPR), 2016, pp. 1571–1576.

[10]

R.K. Sanodiya, J. Mathew, A framework for semi-supervised metric transfer learning on manifolds, Knowl.-Based Syst. 176 (2019) 1–14.

Digital Library

[11]

M. Pérez-Ortiz, P.A. Gutiérrez, M.D. Ayllón-Terán, N. Heaton, R. Ciria, J. Briceño, C. Hervás-Martínez, Synthetic semi-supervised learning in imbalanced domains: constructing a model for donor-recipient matching in liver transplantation, Knowl.-Based Syst. 123 (2017) 75–87.

[12]

K. Kong, J. Lee, Y. Kwak, M. Kang, S.G. Kim, W. Song, Recycling: semi-supervised learning with noisy labels in deep neural networks, IEEE Access 7 (2019) 66998–67005.

[13]

Y. Tian, Y. Zhang, D. Liu, Semi-supervised support vector classification with self-constructed universum, Neurocomputing 189 (2016) 33–42.

[14]

X. Zhu, Z. Ghahramani, J.D. Lafferty, Semi-supervised learning using gaussian fields and harmonic functions, in: T. Fawcett, N. Mishra (Eds.), Machine Learning, Proceedings of the Twentieth International Conference (ICML 2003), August 21–24, 2003, Washington, DC, USA, AAAI Press, 2003, pp. 912–919.

[15]

S. Chi, X. Li, Y. Tian, J. Li, X. Kong, K. Ding, C. Weng, J. Li, Semi-supervised learning to improve generalizability of risk prediction models, J. Biomed. Inform. 92 (2019).

Digital Library

[16]

J. Mo, Y. Gan, H. Yuan, Weighted pseudo labeled data and mutual learning for semi-supervised classification, IEEE Access 9 (2021) 36522–36534.

[17]

Y. Peng, Q. Chen, Z. Lu, An empirical study of multi-task learning on BERT for biomedical text mining, in: D. Demner-Fushman, K.B. Cohen, S. Ananiadou, J. Tsujii (Eds.), Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing, BioNLP 2020, Online, July 9, 2020, Association for Computational Linguistics, 2020, pp. 205–214.

[18]

Ö. Uzuner, A. Stubbs, Practical applications for natural language processing in clinical research: the 2014 i2b2/uthealth shared tasks, J. Biomed. Inform. 58 (2015) S1–S5.

[19]

H. Zhao, N. Ye, R. Wang, Coarse-to-fine speech emotion recognition based on multi-task learning, J. Signal Process. Syst. 93 (2–3) (2021) 299–308.

[20]

D. Fourure, R. Emonet, É. Fromont, D. Muselet, N. Neverova, A. Trémeau, C. Wolf, Multi-task, multi-domain learning: application to semantic segmentation and pose regression, Neurocomputing 251 (2017) 68–80.

[21]

Z. Qi, Y. Tian, Y. Shi, A nonparallel support vector machine for a classification problem with universum learning, J. Comput. Appl. Math. 263 (2014) 288–298.

[22]

C. Zhu, Double-fold localized multiple matrix learning machine with universum, Pattern Anal. Appl. 20 (4) (2017) 1091–1118.

[23]

Ruder, S. : An overview of multi-task learning in deep neural networks. CoRR arXiv:1706.05098.

[24]

Zhang, Y.; Yang, Q. : A survey on multi-task learning. CoRR arXiv:1707.08114.

[25]

Liang Zhao, Yanshan Xiao, Kairun Wen, Bo Liu, Xiangjun Kong: Multi-task manifold learning for partial label learning, Inf. Sci. 602 (2022) 351–365.

[26]

S. Hao, P. Zhao, Y. Liu, S.C.H. Hoi, C. Miao, Online multitask relative similarity learning, in: C. Sierra (Ed.), Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19–25, 2017, ijcai.org, 2017, pp. 1823–1829.

[27]

X. Zhang, X. Zhang, H. Liu, J. Luo, Multi-task clustering with model relation learning, in: J. Lang (Ed.), Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13–19, 2018, Stockholm, Sweden, ijcai.org, 2018, pp. 3132–3140.

[28]

Liang, J.; Liu, Z.; Zhou, J.; Jiang, X.; Zhang, C.; Wang, F. : Model-protected multi-task learning. CoRR arXiv:1809.06546.

[29]

Zhang, J.; Li, Q.; Caselli, R.J.; Ye, J.; Wang, Y. : Multi-task dictionary learning based convolutional neural network for computer aided diagnosis with longitudinal images. CoRR arXiv:1709.00042.

[30]

N. Khosravan, U. Bagci, Semi-supervised multi-task learning for lung cancer diagnosis, in: 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2018, Honolulu, HI, USA, July 18–21, 2018, IEEE, 2018, pp. 710–713.

[31]

J. Zhang, B. Yu, H. Ji, K. Wang, Multi-task feature learning by using trace norm regularization, Open Phys. 15 (1) (2017).

[32]

Y. Ren, X. Yan, Z. Hu, Z. Xu, Self-paced multi-task multi-view capped-norm clustering, in: L. Cheng, A.C. Leung, S. Ozawa (Eds.), Neural Information Processing - Proceedings of the 25th International Conference, Part IV, in: Lecture Notes in Computer Science, vol. 11304, ICONIP 2018, Siem Reap, Cambodia, December 13–16, 2018, Springer, 2018, pp. 205–217.

[33]

Kong, C.; Chen, Y.; Zhang, H.; Yang, L.; Yang, E. : Multitasking framework for unsupervised simple definition generation. CoRR arXiv:2203.12926.

[34]

V. Vapnik, Statistical Learning Theory, Wiley, 1998.

[35]

Z. Qi, Y. Tian, Y. Shi, Twin support vector machine with universum data, Neural Netw. 36 (2012) 112–119.

[36]

B. Richhariya, A. Sharma, M. Tanveer, Improved universum twin support vector machine, in: IEEE Symposium Series on Computational Intelligence, SSCI 2018, Bangalore, India, November 18–21, 2018, IEEE, 2018, pp. 2045–2052.

[37]

S. Dhar, V. Cherkassky, Development and evaluation of cost-sensitive universum-svm, IEEE Trans. Cybern. 45 (4) (2015) 806–818.

[38]

D. Zhang, J. Wang, L. Si, Document clustering with universum, in: W. Ma, J. Nie, R. Baeza-Yates, T. Chua, W.B. Croft (Eds.), Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011, Beijing, China, July 25–29, 2011, ACM, 2011, pp. 873–882.

[39]

X. Chen, H. Yin, F. Jiang, L. Wang, Multi-view dimensionality reduction based on universum learning, Neurocomputing 275 (2018) 2279–2286.

Digital Library

[40]

S. Chen, C. Zhang, Selecting informative universum sample for semi-supervised learning, in: C. Boutilier (Ed.), IJCAI 2009, Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA, July 11–17, 2009, 2009, pp. 1016–1021.

[41]

J. Xu, Q. Wu, J. Zhang, Z. Tang, Exploiting universum data in adaboost using gradient descent, Image Vis. Comput. 32 (8) (2014) 550–557.

[42]

V. Cherkassky, W. Dai, Empirical study of the universum SVM learning for high-dimensional data, in: C. Alippi, M.M. Polycarpou, C.G. Panayiotou, G. Ellinas (Eds.), Artificial Neural Networks - ICANN 2009, Proceedings of the 19th International Conference, Part I, in: Lecture Notes in Computer Science, vol. 5768, Limassol, Cyprus, September 14–17, 2009, Springer, 2009, pp. 932–941.

[43]

Dhar, S.; Ramakrishnan, N.; Cherkassky, V.; Shah, M. : Universum learning for multiclass SVM. CoRR arXiv:1609.09162.

[44]

Dhar, S.; Cherkassky, V. : Universum learning for SVM regression. CoRR arXiv:1605.08497.

[45]

J. Zhao, Y. Xu, A safe sample screening rule for universum support vector machines, Knowl.-Based Syst. 138 (2017) 46–57.

[46]

D. Liu, Y. Tian, R. Bie, Y. Shi, Self-universum support vector machine, Pers. Ubiquitous Comput. 18 (8) (2014) 1813–1819.

[47]

P. Li, S. Chen, Hierarchical gaussian processes model for multi-task learning, Pattern Recognit. 74 (2018) 134–144.

[48]

H. Xing, M. Ji, Robust one-class support vector machine with rescaled hinge loss function, Pattern Recognit. 84 (2018) 152–164.

[49]

B. Liu, H. Xie, Y. Xiao, Multi-task analysis discriminative dictionary learning for one-class learning, Knowl.-Based Syst. 227 (2021).

Recommendations

MultiMatch: Multi-task Learning for Semi-supervised Domain Generalization
Domain generalization (DG) aims at learning a model on source domains to well generalize on the unseen target domain. Although it has achieved great success, most of the existing methods require the label information for all training samples in source ...
Time-frequency based multi-task learning for semi-supervised time series classification
Abstract
Semi-supervised learning is crucial for alleviating labeling burdens in time series classification. Most of the existing semi-supervised time series classification methods extract patterns from the time domain, ignoring the time-...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

In multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Information Sciences: an International Journal

Information Sciences: an International Journal Volume 626, Issue C

May 2023

866 pages

ISSN:0020-0255

Issue’s Table of Contents

Copyright © 2023.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 May 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents