More Web Proxy on the site http://driver.im/

Article

Automatic labeling of multinomial topic models

Authors:

ChengXiang ZhaiAuthors Info & Claims

KDD '07: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 490 - 499

https://doi.org/10.1145/1281192.1281246

Published: 12 August 2007 Publication History

Abstract

Multinomial distributions over words are frequently used to model topics in text collections. A common, major challenge in applying all such topic models to any text mining problem is to label a multinomial topic model accurately so that a user can interpret the discovered topic. So far, such labels have been generated manually in a subjective way. In this paper, we propose probabilistic approaches to automatically labeling multinomial topic models in an objective way. We cast this labeling problem as an optimization problem involving minimizing Kullback-Leibler divergence between word distributions and maximizing mutual information between a label and a topic model. Experiments with user study have been done on two text data sets with different genres.The results show that the proposed labeling methods are quite effective to generate labels that are meaningful and useful for interpreting the discovered topic models. Our methods are general and can be applied to labeling topics learned through all kinds of topic models such as PLSA, LDA, and their variations.

References

[1]

S. Banerjee and T. Pedersen. The design, implementation, and use of the ngram statistics package. pages 370--381, 2003.

Digital Library

[2]

D. Blei and J. Lafferty. Correlated topic models. In NIPS '05: Advances in Neural Information Processing Systems 18, 2005.

[3]

D. M. Blei and J. D. Lafferty. Dynamic topic models. In Proceedings of the 23rd international conference on Machine learning, pages 113--120, 2006.

Digital Library

[4]

D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. J. Mach. Learn. Res., 3:993--1022, 2003.

[5]

J. Carbonell and J. Goldstein. The use of mmr, diversity-based reranking for reordering documents and producing summaries. In Proceedings of SIGIR '98, pages 335--336, 1998.

Digital Library

[6]

J. Chen, J. Yan, B. Zhang, Q. Yang, and Z. Chen. Diverse topic phrase extraction through latent semantic analysis. In Proceedings of ICDM '06, pages 834--838, 2006.

Digital Library

[7]

K. W. Church and P. Hanks. Word association norms, mutual information, and lexicography. Comput. Linguist., 16(1): 22--29, 1990.

Digital Library

[8]

W. B. Croft and J. Lafferty, editors. Language Modeling and Information Retrieval. Kluwer Academic Publishers, 2003.

Digital Library

[9]

T. L. Griffiths and M. Steyvers. Finding scientific topics. Proceedings of the National Academy of Sciences 101(suppl.1): 5228--5235, 2004.

[10]

J. Hammerton, M. Osborne, S. Armstrong, and W. Daelemans. Introduction to special issue on machine learning approaches to shallow parsing. J. Mach. Learn. Res., 2:551--558, 2002.

Digital Library

[11]

T. Hofmann. Probabilistic latent semantic indexing. In Proceedings of ACM SIGIR'99, pages 50--57.

Digital Library

[12]

R. Jin and A. G. Hauptmann. A new probabilistic model for title generation. In Proceedings of the 19th international conference on Computational linguistics, pages 1--7, 2002.

Digital Library

[13]

P. J. Kaufman, Leonard; Rousseeuw. Finding groups in data. an introduction to cluster analysis. Wiley Series in Probability and Mathematical Statistics. Applied Probability and Statistics. Wiley. New York., 1990.

[14]

W. Li and A. McCallum. Pachinko allocation: Dag-structured mixture models of topic correlations. In ICML '06: Proceedings of the 23rd international conference on Machine learning, pages 577--584, 2006.

Digital Library

[15]

C. D. Manning and H. Schtze. Foundations of statistical natural language processing. MIT Press, Cambridge, MA, USA, 1999.

Digital Library

[16]

Q. Mei, C. Liu, H. Su, and C. Zhai. A probabilistic approach to spatiotemporal theme pattern mining on weblogs. In Proceedings of WWW '06, pages 533--542, 2006.

Digital Library

[17]

Q. Mei and C. Zhai. Discovering evolutionary theme patterns from text: an exploration of temporal text mining. In Proceeding of KDD'05, pages 198--207, 2005.

Digital Library

[18]

Q. Mei and C. Zhai. A mixture model for contextual text mining. In Proceedings of KDD '06, pages 649--655, 2006.

Digital Library

[19]

D. Newman, C. Chemudugunta, and P. Smyth. Statistical entity-topic models. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 680--686, 2006.

Digital Library

[20]

P. Pantel and D. Lin. Discovering word senses from text. In Proceedings of KDD '02, pages 613--619, 2002.

Digital Library

[21]

D. R. Radev, E. Hovy, and K. McKeown. Introduction to the special issue on summarization. Comput. Linguist., 28(4): 399--408, 2002.

Digital Library

[22]

M. Steyvers, P. Smyth, M. Rosen-Zvi, and T. Griffiths. Probabilistic author-topic models for information discovery. In Proceedings of KDD'04, pages 306--315, 2004.

Digital Library

[23]

X. Wang and A. McCallum. Topics over time: a non-markov continuous-time model of topical trends. In Proceedings of KDD '06, pages 424--433, 2006.

Digital Library

[24]

X. Wei and W. B. Croft. Lda--based document models for ad-hoc retrieval. In Proceedings of SIGIR '06, pages 178-185, 2006.

Digital Library

[25]

C. Zhai. Fast statistical parsing of noun phrases for document indexing. In Proceedings of the fifth conference on Applied natural language processing, pages 312--319, 1997.

Digital Library

[26]

C. Zhai and J. Lafferty. Model--based feedback in the language modeling approach to information retrieval. In Proceedings of CIKM '01, pages 403--410, 2001.

Digital Library

[27]

C. Zhai, A. Velivelli, and B. Yu. A cross-collection mixture model for comparative text mining. In Proceedings of KDD'04, pages 743--748, 2004.

Digital Library

Cited By

Williams LAnthi EArman LBurnap P(2024)Topic Modelling: Going beyond Token OutputsBig Data and Cognitive Computing10.3390/bdcc80500448:5(44)Online publication date: 25-Apr-2024
https://doi.org/10.3390/bdcc8050044
Yang CKim JKang DEom D(2024)Vision AI System Development for Improved Productivity in Challenging Industrial Environments: A Sustainable and Efficient ApproachApplied Sciences10.3390/app1407275014:7(2750)Online publication date: 25-Mar-2024
https://doi.org/10.3390/app14072750
Wanna SSolovyev NBarron REren MBhattarai MRasmussen KAlexandrov B(2024)TopicTag: Automatic Annotation of NMF Topic Models Using Chain of Thought and Prompt Tuning with LLMsProceedings of the ACM Symposium on Document Engineering 202410.1145/3685650.3685667(1-4)Online publication date: 20-Aug-2024
https://dl.acm.org/doi/10.1145/3685650.3685667
Show More Cited By

Index Terms

Automatic labeling of multinomial topic models
1. Information systems
  1. Information retrieval

Recommendations

Automatic labeling hierarchical topics
CIKM '12: Proceedings of the 21st ACM international conference on Information and knowledge management

Recently, statistical topic modeling has been widely applied in text mining and knowledge management due to its powerful ability. A topic, as a probability distribution over words, is usually difficult to be understood. A common, major challenge in ...
Personalized resource categorisation in folksonomies
MDS '12: Proceedings of the ACM SIGKDD Workshop on Mining Data Semantics

Folksonomies constitute an important type of Web 2.0 services, where users collectively annotate (or "tag") resources to create custom categories. Semantic relation of these categories hint at the possibility of another categorization at a higher level. ...
Topic modeling with network regularization
WWW '08: Proceedings of the 17th international conference on World Wide Web

In this paper, we formally define the problem of topic modeling with network structure (TMN). We propose a novel solution to this problem, which regularizes a statistical topic model with a harmonic regularizer based on a graph structure in the data. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '07: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2007

1080 pages

ISBN:9781595936097

DOI:10.1145/1281192

General Chair:
Pavel Berkhin
Yahoo!, USA
,
Program Chairs:
Rich Caruana
Cornell University, USA
,
Xindong Wu
University of Vermont, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 August 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

KDD07

Sponsor:

KDD07: The 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 12 - 15, 2007

California, San Jose, USA

Acceptance Rates

KDD '07 Paper Acceptance Rate 111 of 573 submissions, 19%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

235
Total Citations
View Citations
2,936
Total Downloads

Downloads (Last 12 months)94
Downloads (Last 6 weeks)12

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Williams LAnthi EArman LBurnap P(2024)Topic Modelling: Going beyond Token OutputsBig Data and Cognitive Computing10.3390/bdcc80500448:5(44)Online publication date: 25-Apr-2024
https://doi.org/10.3390/bdcc8050044
Yang CKim JKang DEom D(2024)Vision AI System Development for Improved Productivity in Challenging Industrial Environments: A Sustainable and Efficient ApproachApplied Sciences10.3390/app1407275014:7(2750)Online publication date: 25-Mar-2024
https://doi.org/10.3390/app14072750
Wanna SSolovyev NBarron REren MBhattarai MRasmussen KAlexandrov B(2024)TopicTag: Automatic Annotation of NMF Topic Models Using Chain of Thought and Prompt Tuning with LLMsProceedings of the ACM Symposium on Document Engineering 202410.1145/3685650.3685667(1-4)Online publication date: 20-Aug-2024
https://dl.acm.org/doi/10.1145/3685650.3685667
Duan GYan SZhang M(2024)A Hybrid Neural Network Model for Sentiment Analysis of Financial Texts Using Topic Extraction, Pre-Trained Model, and Enhanced Attention Mechanism MethodsIEEE Access10.1109/ACCESS.2024.342915012(98207-98224)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3429150
O’Neil SMadlock-Brown CWilkins KMcGrath BDavis HAssaf GWei HZareie PFrench ELoomba JMcMurry JZhou AChute CMoffitt RPfaff EYoo YLeese PChew RLieberman MHaendel M(2024)Finding Long-COVID: temporal topic modeling of electronic health records from the N3C and RECOVER programsnpj Digital Medicine10.1038/s41746-024-01286-37:1Online publication date: 21-Oct-2024
https://doi.org/10.1038/s41746-024-01286-3
Chaudhary AMilios ERajabi E(2024)Top2LabelExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122676242:COnline publication date: 16-May-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122676
Guillén-Pacho IBadenes-Olmedo CCorcho O(2024)Dynamic topic modelling for exploring the scientific literature on coronavirus: an unsupervised labelling techniqueInternational Journal of Data Science and Analytics10.1007/s41060-024-00610-0Online publication date: 13-Aug-2024
https://doi.org/10.1007/s41060-024-00610-0
Olga MLia TMark A(2024)Topic Label Generation in the Popular Science CorpusDigital Geography10.1007/978-3-031-67762-5_9(107-121)Online publication date: 9-Nov-2024
https://doi.org/10.1007/978-3-031-67762-5_9
Yoo MYim SOh SSong J(2023)Solution to Social Problems Using Topic Modeling: From the Perspective of Social Polarization and Multiple DisparitiesJournal of Digital Contents Society10.9728/dcs.2023.24.8.174124:8(1741-1751)Online publication date: 31-Aug-2023
https://doi.org/10.9728/dcs.2023.24.8.1741
Dong XMa JZhang XShaalan AChen QJia S(2023)Revolution trend investigation of tourism destination image with machine learningJournal of Vacation Marketing10.1177/13567667231213152Online publication date: 9-Nov-2023
https://doi.org/10.1177/13567667231213152
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents