More Web Proxy on the site http://driver.im/

Article

Cost-sensitive semi-supervised support vector machine

Authors:

Zhi-Hua ZhouAuthors Info & Claims

AAAI'10: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence

Pages 500 - 505

Published: 11 July 2010 Publication History

Abstract

In this paper, we study cost-sensitive semi-supervised learning where many of the training examples are un-labeled and different misclassification errors are associated with unequal costs. This scenario occurs in many real-world applications. For example, in some disease diagnosis, the cost of erroneously diagnosing a patient as healthy is much higher than that of diagnosing a healthy person as a patient. Also, the acquisition of labeled data requires medical diagnosis which is expensive, while the collection of unlabeled data such as basic health information is much cheaper. We propose the CS4VM (Cost-Sensitive Semi-Supervised Support Vector Machine) to address this problem. We show that the CS4VM, when given the label means of the unlabeled data, closely approximates the supervised cost-sensitive SVM that has access to the ground-truth labels of all the unlabeled data. This observation leads to an efficient algorithm which first estimates the label means and then trains the CS4VM with the plug-in label means by an efficient SVM solver. Experiments on a broad range of data sets show that the proposed method is capable of reducing the total cost and is computationally efficient.

References

[1]

Belkin, M.; Niyogi, P.; and Sindhwani, V. 2006. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7:2399-2434.

[2]

Bennett, K., and Demiriz, A. 1999. Semi-supervised support vector machines. In Adv. Neural Infor. Process. Syst. 11. 368-374.

[3]

Brefeld, U.; Geibel, P.; and Wysotzki, F. 2003. Support vector machines with example dependent costs. In Proc. 14th Eur. Conf. Mach. Learn., 23-34.

[4]

Chapelle, O., and Zien, A. 2005. Semi-supervised learning by low density separation. In Proc. 10th Int'l Workshop AI and Stat., 57-64.

[5]

Chapelle, O.; Schölkopf, B.; and Zien, A., eds. 2006. Semi-Supervised Learning. Cambridge, MA: MIT Press.

[6]

Chapelle, O.; Sindhwani, V.; and Keerthi, S. S. 2008. Optimization techniques for semi-supervised support vector machines. J. Mach. Learn. Res. 9:203-233.

[7]

Collobert, R.; Sinz, F.; Weston, J.; and Bottou, L. 2006. Large scale transductive SVMs. J. Mach. Learn. Res. 7:1687-1712.

[8]

Domingos, P. 1999. MetaCost: A general method for making classifiers cost-sensitive. In Proc. 5th ACM SIGKDD Int'l Conf. Knowl. Disc. Data Min., 155-164.

[9]

Elkan, C. 2001. The foundations of cost-sensitive learning. In Proc. 17th Int'l Joint Conf. Artif. Intell., 973-978.

[10]

Fan, W.; Stolfo, S. J.; Zhang, J.; and Chan, P. K. 1999. AdaCost: Misclassification cost-sensitive boosting. In Proc. 16th Int'l Conf. Mach. Learn., 97-104.

[11]

Greiner, R.; Grove, A. J.; and Roth, D. 2002. Learning cost-sensitive active classifiers. Artif. Intell. 139(2):137-174.

[12]

Gretton, A.; Borgwardt, K. M.; Rasch, M.; Schölkopf, B.; and Smola, A. J. 2006. A kernel method for the two-sample-problem. In Adv. Neural Infor. Process. Syst. 19. 513-520.

[13]

Joachims, T. 1999. Transductive inference for text classification using support vector machines. In Proc. 16th Int'l Conf. Mach. Learn., 200-209.

[14]

Kukar, M., and Kononenko, I. 1998. Cost-sensitive learning with neural networks. In Proc. 13th Eur. Conf. Artif. Intell., 445-449.

[15]

Lee, Y.; Lin, Y.; and Wahba, G. 2004. Multicategory support vector machines: Theory and application to the classification of microarray data and satellite radiance data. J. Ame. Stat. Assoc. 99(465):67-82.

[16]

Li, Y.-F.; Kwok, J. T.; and Zhou, Z.-H. 2009. Semi-supervised learning using label mean. In Proc. 26th Int'l Conf. Mach. Learn., 633-640.

[17]

Liu, A.; Jun, G.; and Ghosh, J. 2009. Spatially cost-sensitive active learning. In Proc. 9th SIAM Int'l Conf. Data Min., 814-825.

[18]

Lozano, A. C., and Abe, N. 2008. Multi-class cost-sensitive boosting with p-norm loss functions. In Proc. 14th ACM SIGKDD Int'l Conf. Knowl. Disc. Data Min., 506-514.

[19]

Margineantu, D. D. 2005. Active cost-sensitive learning. In Proc. 19th Int'l Joint Conf. Artif. Intell., 1622-1623.

[20]

Masnadi-Shirazi, H., and Vasconcelos, N. 2007. Asymmetric boosting. In Proc. 24th Int'l Conf. Mach. Learn., 619-626.

[21]

Morik, K.; Brockhausen, P.; and Joachims, T. 1999. Combining statistical learning with a knowledge-based approach: A case study in intensive care monitoring. In Proc. 16th Int'l Conf. Mach. Learn., 268-277.

[22]

Qin, Z.; Zhang, S.; Liu, L.; and Wang, T. 2008. Cost-sensitive semi-supervised classification using CS-EM. In Proc. 8th Int'l Conf. Comput. Infor. Tech., 131-136.

[23]

Ting, K. M. 2002. An instance-weighting method to induce cost-sensitive trees. IEEE Trans Knowl. Data Eng. 14(3):659-665.

[24]

Turney, P. 2000. Types of cost in inductive concept learning. In ICML Workshop Cost-Sensitive Learning, 15-21.

[25]

Zadrozny, B.; Langford, J.; and Abe, N. 2003. Cost-sensitive learning by cost-proportionate example weighting. In Proc. 3rd IEEE Int'l Conf. Data Min., 435-442.

[26]

Zhou, Z.-H., and Li, M. 2010. Semi-supervised learning by disagreement. Knowl. Infor. Syst.

[27]

Zhou, Z.-H., and Liu, X.-Y. 2006a. On multi-class cost-sensitive learning. In Proc. 21st Nat'l Conf. Artif. Intell., 567-572.

[28]

Zhou, Z.-H., and Liu, X.-Y. 2006b. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans Knowl. Data Eng. 18(1):63-77.

[29]

Zhu, X. 2007. Semi-supervised learning literature survey. Technical report, Department of Computer Science, University of Wisconsin-Madison.

Cited By

Zhang TZhou Z(2018)Semi-supervised optimal margin distribution machinesProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304889.3305092(3104-3110)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304889.3305092
Qian PXi CXu MJiang YSu KWang SMuzic R(2018)SSC-EKEInformation Sciences: an International Journal10.1016/j.ins.2017.08.093422:C(51-76)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1016/j.ins.2017.08.093
Yang ZXu Y(2018)A safe screening rule for Laplacian support vector machineEngineering Applications of Artificial Intelligence10.1016/j.engappai.2017.10.01167:C(309-316)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1016/j.engappai.2017.10.011
Show More Cited By

Cost-sensitive semi-supervised support vector machine
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning

Recommendations

Semi-supervised support vector machines
NIPS'98: Proceedings of the 12th International Conference on Neural Information Processing Systems

We introduce a semi-supervised support vector machine (S³VM) method. Given a training set of labeled data and a working set of unlabeled data, S³VM constructs a support vector machine using both the training and working sets. We use S³VM to solve the ...
An overview on semi-supervised support vector machine

Support vector machine (SVM) is a machine learning method based on statistical learning theory. It has a lot of advantages, such as solid theoretical foundation, global optimization, the sparsity of the solution, nonlinear and generalization. The ...
Semi-supervised learning combining transductive support vector machine with active learning

In typical data mining applications, labeling the large amounts of data is difficult, expensive, and time consuming, if annotated manually. To avoid manual labeling, semi-supervised learning uses unlabeled data along with the labeled data in the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'10: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence

July 2010

1970 pages

Publisher

AAAI Press

Publication History

Published: 11 July 2010

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang TZhou Z(2018)Semi-supervised optimal margin distribution machinesProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304889.3305092(3104-3110)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304889.3305092
Qian PXi CXu MJiang YSu KWang SMuzic R(2018)SSC-EKEInformation Sciences: an International Journal10.1016/j.ins.2017.08.093422:C(51-76)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1016/j.ins.2017.08.093
Yang ZXu Y(2018)A safe screening rule for Laplacian support vector machineEngineering Applications of Artificial Intelligence10.1016/j.engappai.2017.10.01167:C(309-316)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1016/j.engappai.2017.10.011
Altnel BCan Ganiz MDiri B(2017)Instance labeling in semi-supervised learning with meaning values of wordsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2017.04.00362:C(152-163)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1016/j.engappai.2017.04.003
Ding SZhu ZZhang X(2017)An overview on semi-supervised support vector machineNeural Computing and Applications10.1007/s00521-015-2113-728:5(969-978)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1007/s00521-015-2113-7
Yan YXu ZTsang ILong GYang Y(2016)Robust semi-supervised learning through label aggregationProceedings of the Thirtieth AAAI Conference on Artificial Intelligence10.5555/3016100.3016212(2244-2250)Online publication date: 12-Feb-2016
https://dl.acm.org/doi/10.5555/3016100.3016212
Liu ZGao CYang HHe Q(2016)A Cost-Sensitive Sparse Representation Based Classification for Class-Imbalance ProblemScientific Programming10.1155/2016/80350892016(8)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1155/2016/8035089
Zhang GSun HJi ZYuan YSun Q(2016)Cost-sensitive dictionary learning for face recognitionPattern Recognition10.1016/j.patcog.2016.06.01260:C(613-629)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1016/j.patcog.2016.06.012
Altınel BGaniz M(2016)A new hybrid semi-supervised algorithm for text classification with class-based semanticsKnowledge-Based Systems10.1016/j.knosys.2016.06.021108:C(50-64)Online publication date: 15-Sep-2016
https://dl.acm.org/doi/10.1016/j.knosys.2016.06.021
Moraes DWainer JRocha A(2016)Low false positive learning with support vector machinesJournal of Visual Communication and Image Representation10.1016/j.jvcir.2016.03.00738:C(340-350)Online publication date: 1-Jul-2016
https://dl.acm.org/doi/10.1016/j.jvcir.2016.03.007
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents