More Web Proxy on the site http://driver.im/

Article

Text classification in a hierarchical mixture model for small training sets

Authors:

Kristina Toutanova,

Thomas HofmannAuthors Info & Claims

CIKM '01: Proceedings of the tenth international conference on Information and knowledge management

Pages 105 - 113

https://doi.org/10.1145/502585.502604

Published: 05 October 2001 Publication History

Abstract

Documents are commonly categorized into hierarchies of topics, such as the ones maintained by Yahoo! and the Open Directory project, in order to facilitate browsing and other interactive forms of information retrieval. In addition, topic hierarchies can be utilized to overcome the sparseness problem in text categorization with a large number of categories, which is the main focus of this paper. This paper presents a hierarchical mixture model which extends the standard naive Bayes classifier and previous hierarchical approaches. Improved estimates of the term distributions are made by differentiation of words in the hierarchy according to their level of generality/specificity. Experiments on the Newsgroups and the Reuters-21578 dataset indicate improved performance of the proposed classifier in comparison to other state-of-the-art methods on datasets with a small number of positive examples.

References

[1]

S. D'Alessio, M. Murray, R. Schiaflino, and A. Kershenbaum. Category levels in hierarchical text categorization. In Proceedingf of EMNLP-3, 3rd Conference on Empirical Methods in Natural Language Processing, 1998.

[2]

S. Dumais and H. Chen. Hierarchical classification of web content. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 256-263, Athens, Greece, August 2000.

Digital Library

[3]

S. Dumais, J. Platt, D. Heckerman, and M. Sahami. Inductive learning algorithms and representations for text categorization. In Proceedings of the International Conference on Information and Knowledge Management, pages 148-155, 1998.

Digital Library

[4]

T. Hofmann. The cluster-abstraction model: unsupervised learning of topic hierarchies from text data. In Proceedings of the International Joint Conference in Artificial Intelligence, pages 6822687, 1999.

Digital Library

[5]

T. Hofmann. Probabilistic latent semantic indexing. In Proceedings of the 22nd International Conference on Research and Development in Information Retrieval (SIGIR '99) pages 50-57, Berkeley, California, August 1999.

Digital Library

[6]

T. Hofmann and J. Puzicha. Statistical models for cooccmrence data. AI-MEMO 1625, Artifical Intelligence Laboratory, Massachusetts Institute of Technology, 1998.

Digital Library

[7]

F. Jelinek and R. Mercer. Interpolated estimation of Markov source parameters from sparse data. In S. Gelsema and L. Kanal, editors, Pattern Recognition in Practice, pages 381-402. North-Holland, 1980.

[8]

T. Joachims. Making large-scale svm learning practical. In B. Scholkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning. MIT Press, 1999.

Digital Library

[9]

D. Koller and M. Sahami. Hierarchically classifying documents using very few words. In Proceedings of the International Conference on Machine Learning, pages 170-178, 1997.

Digital Library

[10]

K. Lang. Newsweeder: Learning to filter netnews. In International Conference on Machine Learning, pages 331-339, 1995.

[11]

D. D. Lewis. Naive (bayes) at forty: The independence assumption in information retrieval. In Proceedings of the 1998 European Conference on Machine Learning, 1998.

Digital Library

[12]

C. Manning and H. Schuetze. Foundations of Statistical Natural Language Processing. MIT Press, 2000.

Digital Library

[13]

A. McCallum and K. Nigam. A comparison of event models for naive Bayes text classification. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-93), 1998.

[14]

A. McCallum, R. Rosenfeld, T. Mitchell, and A. Ng. Improving text classification by shrinkage in a hierarchy of classes. In Proceedings of the Fifteenth International Conference on Machine Learning, pages 359-367, Madison, Wisconsin, 1998.

Digital Library

[15]

T. Mitchell. Machine Learning. McGraw Hill, 1997.

Digital Library

[16]

K. Nigam, A. McCallum, S. Thrun, and T. Mitchell. Learning to classify text from labeled and unlabeled documents. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, 1998.

Digital Library

[17]

A. Weigend, E. Wiener, and J. Pedersen. Exploiting hierarchy in text categorization. Information Retrieval, pages 193-216, 1999.

Digital Library

[18]

Y. Yang and X. Liu. A re-examination of text categorization methods. In Proceedings of the 22nd International Conference on Research and Development in Information Retrieval (SIGIR'99), pages 4249, Berkley, August 1999.

Digital Library

Cited By

Fredriksson TMattos DBosch JOlsson H(2021)Assessing the Suitability of Semi-Supervised Learning Datasets using Item Response Theory2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA53835.2021.00049(326-333)Online publication date: Sep-2021
https://doi.org/10.1109/SEAA53835.2021.00049
Rexha ADragoni MKern RHuang RWu DMarchionini GHe DCunningham SHansen P(2020)A Neural-based Architecture For Small Datasets ClassificationProceedings of the ACM/IEEE Joint Conference on Digital Libraries in 202010.1145/3383583.3398535(319-327)Online publication date: 1-Aug-2020
https://dl.acm.org/doi/10.1145/3383583.3398535
Kolluri JRazia S(2020)Text classification using Naïve Bayes classifierMaterials Today: Proceedings10.1016/j.matpr.2020.10.058Online publication date: Nov-2020
https://doi.org/10.1016/j.matpr.2020.10.058
Show More Cited By

Index Terms

Text classification in a hierarchical mixture model for small training sets
1. Applied computing
  1. Document management and text processing
    1. Document capture
      1. Document analysis
2. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Language resources
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees

Recommendations

Hierarchical Multi-label Text Classification: An Attention-based Recurrent Network Approach
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Hierarchical multi-label text classification (HMTC) is a fundamental but challenging task of numerous applications (e.g., patent annotation), where documents are assigned to multiple categories stored in a hierarchical structure. Categories at different ...
Classification Performance of Bagging and Boosting Type Ensemble Methods with Small Training Sets
Abstract
Classification performance of an ensemble method can be deciphered by studying the bias and variance contribution to its classification error. Statistically, the bias and variance of a single classifier is controlled by the size of the training ...
Active learning for hierarchical text classification
PAKDD'12: Proceedings of the 16th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part I

Hierarchical text classification plays an important role in many real-world applications, such as webpage topic classification, product categorization and user feedback classification. Usually a large number of training examples are needed to build an ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '01: Proceedings of the tenth international conference on Information and knowledge management

October 2001

616 pages

ISBN:1581134363

DOI:10.1145/502585

Editors:
Henrique Paques
Georgia Institute of Technology
,
Ling Liu
Georgia Institute of Technology
,
David Grossman
Illinois Institute of Technology
,
General Chair:
Calton Pu
Georgia Institute of Technology

Copyright © 2001 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 October 2001

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

CIKM01

Sponsor:

CIKM01: International Conference on Information and Knowledge Management

October 5 - 10, 2001

Georgia, Atlanta, USA

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
1,213
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fredriksson TMattos DBosch JOlsson H(2021)Assessing the Suitability of Semi-Supervised Learning Datasets using Item Response Theory2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA53835.2021.00049(326-333)Online publication date: Sep-2021
https://doi.org/10.1109/SEAA53835.2021.00049
Rexha ADragoni MKern RHuang RWu DMarchionini GHe DCunningham SHansen P(2020)A Neural-based Architecture For Small Datasets ClassificationProceedings of the ACM/IEEE Joint Conference on Digital Libraries in 202010.1145/3383583.3398535(319-327)Online publication date: 1-Aug-2020
https://dl.acm.org/doi/10.1145/3383583.3398535
Kolluri JRazia S(2020)Text classification using Naïve Bayes classifierMaterials Today: Proceedings10.1016/j.matpr.2020.10.058Online publication date: Nov-2020
https://doi.org/10.1016/j.matpr.2020.10.058
Fadziso T(2019)An Approach to Enhance Text Categorization through Shrinkage in a Hierarchy of ModulesABC Journal of Advanced Research10.18034/abcjar.v8i2.5628:2(123-130)Online publication date: 31-Dec-2019
https://doi.org/10.18034/abcjar.v8i2.562
Vandic DFrasincar FKaymak U(2018)A framework for product description classification in e-commerceJournal of Web Engineering10.5555/3370048.337004917:1-2(1-27)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.5555/3370048.3370049
Puurula AMyaeng SCulpepper SZuccon GSitbon L(2013)Integrated instance- and class-based generative modeling for text classificationProceedings of the 18th Australasian Document Computing Symposium10.1145/2537734.2537751(66-73)Online publication date: 5-Dec-2013
https://dl.acm.org/doi/10.1145/2537734.2537751
Swezey RShiramatsu SOzono TShintani T(2012)An Improvement for Naive Bayes Text Classification Applied to Online Imbalanced Crowdsourced CorpusesModern Advances in Intelligent Systems and Tools10.1007/978-3-642-30732-4_19(147-152)Online publication date: 2012
https://doi.org/10.1007/978-3-642-30732-4_19
de Colla Furquim Lde Lima V(2012)Clustering and categorization of Brazilian portuguese legal documentsProceedings of the 10th international conference on Computational Processing of the Portuguese Language10.1007/978-3-642-28885-2_31(272-283)Online publication date: 17-Apr-2012
https://dl.acm.org/doi/10.1007/978-3-642-28885-2_31
Garcia-Constantino MCoenen FNoble PRadford A(2012)Questionnaire Free Text Summarisation Using Hierarchical ClassificationResearch and Development in Intelligent Systems XXIX10.1007/978-1-4471-4739-8_3(35-48)Online publication date: 9-Oct-2012
https://doi.org/10.1007/978-1-4471-4739-8_3
Cerri Rde Carvalho AFreitas A(2011)Adapting non-hierarchical multilabel classification methods for hierarchical multilabel classificationIntelligent Data Analysis10.5555/2595490.259549415:6(861-887)Online publication date: 1-Nov-2011
https://dl.acm.org/doi/10.5555/2595490.2595494
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents