More Web Proxy on the site http://driver.im/

research-article

Quantification and semi-supervised classification methods for handling changes in class distribution

Authors:

Jack Chongjie Xue,

Gary M. WeissAuthors Info & Claims

KDD '09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 897 - 906

https://doi.org/10.1145/1557019.1557117

Published: 28 June 2009 Publication History

Abstract

In realistic settings the prevalence of a class may change after a classifier is induced and this will degrade the performance of the classifier. Further complicating this scenario is the fact that labeled data is often scarce and expensive. In this paper we address the problem where the class distribution changes and only unlabeled examples are available from the new distribution. We design and evaluate a number of methods for coping with this problem and compare the performance of these methods. Our quantification-based methods estimate the class distribution of the unlabeled data from the changed distribution and adjust the original classifier accordingly, while our semi-supervised methods build a new classifier using the examples from the new (unlabeled) distribution which are supplemented with predicted class values. We also introduce a hybrid method that utilizes both quantification and semi-supervised learning. All methods are evaluated using accuracy and F-measure on a set of benchmark data sets. Our results demonstrate that our methods yield substantial improvements in accuracy and F-measure.

Supplementary Material

JPG File (p897-weiss.jpg)

Download
8.09 KB

MP4 File (p897-weiss.mp4)

Download
71.15 MB

References

[1]

Alaiz-Rodriguez, R., Guerrero-Curieses, A. and Cid-Sueiro, J. Minimax Regret Classifier for Imprecise Class Distributions. Journal of Machine Learning Research, 8 (2007), 103--130.

Digital Library

[2]

Alaiz-Rodriguez, R. and Japkowicz, N. Assessing the Impact of Changing Environments on Classifier Performance. In the Proceedings of the 21st Canadian Conference in Artificial Intelligence (2008), 13--24.

Digital Library

[3]

Asuncion, A. and Newman, D. J. UCI Machine Learning Repository. http://www.ics.uci.edu/~mlearn/ MLRepository.html (2007).

[4]

Chapelle, O., Scholkopf, B. and Zien, A. Semi-Supervised Learning. MIT Press, Cambridge, MA, 2006.

Digital Library

[5]

Chawla, N. V., Bowyer, K. W. and Kegelmeyer, W. P. SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16 (2002), 321--357.

Digital Library

[6]

Elkan, C. The foundations of cost-sensitive learning. In the Proceedings of the 17th International Joint Conference on Artificial Intelligence (2001), 973--978.

Digital Library

[7]

Forman, G. Quantifying counts and costs via classification. Data Mining Knowledge Discovery, 17, 2 (2008), 164--206.

Digital Library

[8]

Forman, G. Counting Positives Accurately Despite Inaccurate Classification. In ECML (2005), 564--575.

Digital Library

[9]

Latinne, P., Saerens, M. and Decaestecker, C. Adjusting the Outputs of a Classifier to New a Priori Probabilities May Significantly Improve Classification Accuracy: Evidence from a multi-class problem in remote sensing. In the Proceedings of the 18th International Conference on Machine Learning (2001), 298--305.

Digital Library

[10]

Provost, F. and Fawcett, T. Robust Classification for Imprecise Environments. Machine Learning, 42, 3 (2001), 203--231.

Digital Library

[11]

Provost, F., Fawcett, T. and Kohavi, R. The Case against Accuracy Estimation for Comparing Induction Algorithms. In the Proceedings of the 15th International Conference on Machine Learning (1998), 445--453.

Digital Library

[12]

Quinlan, R. J. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.

Digital Library

[13]

Saerens, M., Latinne, P. and Decaestecker, C. Adjusting the outputs of a classifier to new a priori probabilities: A simple procedure. Neural Computing, 14 (2002), 14--21.

Digital Library

[14]

Tsymbal, A. The problem of concept drift: Definitions and related work. Computer Science Department, Trinity College Dublin, 2004.

[15]

Weiss, G. M. Mining with rarity: a unifying framework. SIGKDD Explorations Newsletter, 6, 1 (2004), 7--19.

Digital Library

[16]

Weiss, G. M. and Provost, F. Learning when training data are costly: The effect of class distribution on tree induction. Journal of Artificial Intelligence Research, 19 (2003), 315--354.

Digital Library

[17]

Witten, I. H. and Frank, E. Data Mining: Practical Machine Learning Tools and Techniques, 2nd ed. Morgan Kaufmann, June 2005.

Digital Library

[18]

Yang, C. and Zhou, J. Non-stationary data sequence classification using online class priors estimation. Pattern Recognition, 41, 8 (2008), 2656--2664.

Digital Library

[19]

Zadrozny, B. and Elkan, C. Learning and making decisions when costs and probabilities are both unknown. In the Proceedings of the 7th International Conference on Knowledge Discovery and Data Mining (2001), 204--213.

Digital Library

[20]

Zhu, X. Semi-Supervised Learning Literature Survey. Computer Science Department, University of Wisconsin-Madison, 2005.

Cited By

Donyavi ZSerapião ABatista G(2024)MC-SQ and MC-MQ: Ensembles for Multi-Class QuantificationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.337201136:8(4007-4019)Online publication date: Aug-2024
https://doi.org/10.1109/TKDE.2024.3372011
Malinda DSenarathna DTragoudas S(2024)Enhanced Distribution Matching for Multiclass Quantification2024 International Conference on Machine Learning and Applications (ICMLA)10.1109/ICMLA61862.2024.00192(1238-1242)Online publication date: 18-Dec-2024
https://doi.org/10.1109/ICMLA61862.2024.00192
Esuli AFabris AMoreo ASebastiani FEsuli AFabris AMoreo ASebastiani F(2022)The Road AheadLearning to Quantify10.1007/978-3-031-20467-8_7(121-123)Online publication date: 29-Dec-2022
https://doi.org/10.1007/978-3-031-20467-8_7
Show More Cited By

Recommendations

A Review on Quantification Learning

The task of quantification consists in providing an aggregate estimation (e.g., the class distribution in a classification problem) for unseen test sets, applying a model that is trained using a training set with a different data distribution. Several ...
Quantifying trends accurately despite classifier error and class imbalance
KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining

This paper promotes a new task for supervised machine learning research: quantification - the pursuit of learning methods for accurately estimating the class distribution of a test set, with no concern for predictions on individual cases. A variant for ...
Deep learning at 15PF: supervised and semi-supervised classification for scientific data
SC '17: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

This paper presents the first, 15-PetaFLOP Deep Learning system for solving scientific pattern classification problems on contemporary HPC architectures. We develop supervised convolutional architectures for discriminating signals in high-energy physics ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining

June 2009

1426 pages

ISBN:9781605584959

DOI:10.1145/1557019

General Chairs:
John Elder
Elder Research, Inc., USA
,
Françoise Soulié Fogelman
KXEN, France
,
Program Chairs:
Peter Flach
University of Bristol, UK
,
Mohammed Zaki
RPI, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD09

Sponsor:

KDD09: The 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

June 28 - July 1, 2009

Paris, France

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
763
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)9

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Donyavi ZSerapião ABatista G(2024)MC-SQ and MC-MQ: Ensembles for Multi-Class QuantificationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.337201136:8(4007-4019)Online publication date: Aug-2024
https://doi.org/10.1109/TKDE.2024.3372011
Malinda DSenarathna DTragoudas S(2024)Enhanced Distribution Matching for Multiclass Quantification2024 International Conference on Machine Learning and Applications (ICMLA)10.1109/ICMLA61862.2024.00192(1238-1242)Online publication date: 18-Dec-2024
https://doi.org/10.1109/ICMLA61862.2024.00192
Esuli AFabris AMoreo ASebastiani FEsuli AFabris AMoreo ASebastiani F(2022)The Road AheadLearning to Quantify10.1007/978-3-031-20467-8_7(121-123)Online publication date: 29-Dec-2022
https://doi.org/10.1007/978-3-031-20467-8_7
Esuli AFabris AMoreo ASebastiani FEsuli AFabris AMoreo ASebastiani F(2022)The Quantification LandscapeLearning to Quantify10.1007/978-3-031-20467-8_6(103-120)Online publication date: 29-Dec-2022
https://doi.org/10.1007/978-3-031-20467-8_6
Esuli AFabris AMoreo ASebastiani FEsuli AFabris AMoreo ASebastiani F(2022)Advanced TopicsLearning to Quantify10.1007/978-3-031-20467-8_5(87-101)Online publication date: 29-Dec-2022
https://doi.org/10.1007/978-3-031-20467-8_5
Esuli AFabris AMoreo ASebastiani FEsuli AFabris AMoreo ASebastiani F(2022)Methods for Learning to QuantifyLearning to Quantify10.1007/978-3-031-20467-8_4(55-85)Online publication date: 29-Dec-2022
https://doi.org/10.1007/978-3-031-20467-8_4
Esuli AFabris AMoreo ASebastiani FEsuli AFabris AMoreo ASebastiani F(2022)Evaluation of Quantification AlgorithmsLearning to Quantify10.1007/978-3-031-20467-8_3(33-54)Online publication date: 29-Dec-2022
https://doi.org/10.1007/978-3-031-20467-8_3
Esuli AFabris AMoreo ASebastiani FEsuli AFabris AMoreo ASebastiani F(2022)Applications of QuantificationLearning to Quantify10.1007/978-3-031-20467-8_2(19-31)Online publication date: 29-Dec-2022
https://doi.org/10.1007/978-3-031-20467-8_2
Esuli AFabris AMoreo ASebastiani FEsuli AFabris AMoreo ASebastiani F(2022)The Case for QuantificationLearning to Quantify10.1007/978-3-031-20467-8_1(1-17)Online publication date: 29-Dec-2022
https://doi.org/10.1007/978-3-031-20467-8_1
Daughton APaul M(2021)A bootstrapping approach to social media quantificationSocial Network Analysis and Mining10.1007/s13278-021-00760-011:1Online publication date: 9-Aug-2021
https://doi.org/10.1007/s13278-021-00760-0
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten