More Web Proxy on the site http://driver.im/

article

Evaluating WordNet-based Measures of Lexical Semantic Relatedness

Authors:

Alexander Budanitsky,

Graeme HirstAuthors Info & Claims

Computational Linguistics, Volume 32, Issue 1

Pages 13 - 47

https://doi.org/10.1162/coli.2006.32.1.13

Published: 01 March 2006 Publication History

Abstract

The quantification of lexical semantic relatedness has many applications in NLP, and many different measures have been proposed. We evaluate five of these measures, all of which use WordNet as their central resource, by comparing their performance in detecting and correcting real-word spelling errors. An information-content-based measure proposed by Jiang and Conrath is found superior to those proposed by Hirst and St-Onge, Leacock and Chodorow, Lin, and Resnik. In addition, we explain why distributional similarity is not an adequate proxy for lexical semantic relatedness.

References

[1]

Agresti, Alan and Barbara Finlay. 1997. Statistical Methods for the Social Sciences. Prentice Hall, Upper Saddle River, NJ, 3rd edition.

[2]

Banerjee, Satanjeev and Ted Pedersen. 2003. Extended gloss overlaps as a measure of semantic relatedness. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, pages 805-810, August.

[3]

Barsalou, Lawrence W. 1983. Ad hoc categories. Memory and Cognition, 11:211-227.

[4]

Barsalou, Lawrence W. 1989. Intra-concept similarity and its implications for interconcept similarity. In Stella Vosniadou and Andrew Ortony, editors, Similarity and Analogical Reasoning. Cambridge University Press, pages 76-121.

[5]

Bernard, J. R. L., editor. 1986. The Macquarie Thesaurus. Macquarie Library, Sydney, Australia.

[6]

Budanitsky, Alexander. 1999. Lexical semantic relatedness and its application in natural language processing. Technical Report CSRG-390, Computer Systems Research Group, University of Toronto, August.

[7]

Budanitsky, Alexander and Graeme Hirst. 2001. Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures. In Workshop on WordNet and Other Lexical Resources, Second Meeting of the North American Chapter of the Association for Computational Linguistics, Pittsburgh, PA, pages 29-34.

[8]

Corley, Courtney and Rada Mihalcea. 2005. Measuring the semantic similarity of texts. In Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment, Ann Arbor, MI, June, pages 13-18.

[9]

Dagan, Ido. 2000. Contextual word similarity. In Robert Dale, Hermann Moisl, and Harold Somers, editors, Handbook of Natural Language Processing. Marcel Dekker, pages 459-475.

[10]

Dagan, Ido, Lillian Lee, and Fernando C. N. Pereira. 1999. Similarity-based models of word cooccurrence probabilities. Machine Learning, 34(1-3):43-69.

[11]

Fellbaum, Christiane, editor. 1998. WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA.

[12]

Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2002. Placing search in context: The concept revisited. ACM Transactions on Information Systems, 20:116-131.

[13]

Francis,Winthrop Nelson and Henry Ku¿era. 1982. Frequency Analysis of English Usage: Lexicon and Grammar. Houghton Mifflin, Boston.

[14]

Fromkin, Victoria A. 1980. Errors in Linguistic performance: Slips of the Tongue, Ear, Pen, and Hand. Academic Press, San Francisco.

[15]

Grefenstette, Gregory. 1994. Explorations in Automatic Thesaurus Discovery. Kluwer Academic Publishers, Norwell, MA.

[16]

Halliday, M. A. K. and Ruqaiya Hasan. 1976. Cohesion in English. English Language Series. Longman, New York.

[17]

Hirst, Graeme and Alexander Budanitsky. 2005. Correcting real-word spelling errors by restoring lexical cohesion. Natural Language Engineering, 11:87-111.

[18]

Hirst, Graeme and David St-Onge. 1998. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 305-332.

[19]

Hoey, Michael. 1991. Patterns of Lexis in Text. Describing English Language. Oxford University Press.

[20]

Jarmasz, Mario and Stan Szpakowicz. 2003. Roget's Thesaurus and semantic similarity. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP-2003), Borovetz, Bulgaria, pages 212-219, September.

[21]

Jiang, Jay J. and David W. Conrath. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pages 19-33.

[22]

Kohomban, Upali Sathyajith and Wee Sun Lee. 2005. Learning semantic classes for word sense disambiguation. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05), pages 34-41, Ann Arbor, MI, June.

[23]

Kozima, Hideki and Teiji Furugori. 1993. Similarity between words computed by spreading activation on an English dictionary. In Proceedings of 6th Conference of the European Chapter of the Association for Computational Linguistics (EACL-93), pages 232-239, Utrecht.

[24]

Kozima, Hideki and Akira Ito. 1997. Context-sensitive word distance by adaptive scaling of a semantic space. In Ruslan Mitkov and Nicolas Nicolov, editors, Recent Advances in Natural Language Processing: Selected Papers from RANLP'95, volume 136 of Amsterdam Studies in the Theory and History of Linguistic Science: Current Issues in Linguistic Theory. John Benjamins Publishing Company, Amsterdam/Philadelphia, chapter 2, pages 111-124.

[25]

Lakoff, George. 1987. Women, Fire, and Dangerous Things: What Categories Reveal About the Mind. University of Chicago Press, Chicago.

[26]

Leacock, Claudia and Martin Chodorow. 1998. Combining local context and WordNet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 265-283.

[27]

Lee, Joon Ho, Myong Ho Kim, and Yoon Joon Lee. 1993. Information retrieval based on conceptual distance in IS-A hierarchies. Journal of Documentation, 49(2):188-207.

[28]

Lee, Lillian. 1999. Measures of distributional similarity. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL-99), College Park, MD, pages 25-31.

[29]

Lesk, Michael. 1986. Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from an ice cream cone. In Proceedings, Fifth International Conference on Systems Documentation (SIGDOC '86), Toronto, Canada, pages 24-26.

[30]

Lin, Dekang. 1998a. Automatic retrieval and clustering of similar words. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics (COLING-ACL '98), Montreal, Canada, pages 768-774, August.

[31]

Lin, Dekang. 1998b. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, Madison, WI, pages 296-304, July.

[32]

McCarthy, Diana, Rob Koeling, Julie Weeds, and John Carroll. 2004. Finding predominant word senses in untagged text. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain, pages 280-287.

[33]

Miller, George A. and Walter G. Charles. 1991. Contextual correlates of semantic similarity. Language and Cognitive Processes, 6(1):1-28.

[34]

Mohammad, Saif and Graeme Hirst. 2005. Distributional measures as proxies for semantic relatedness. Submitted for publication.

[35]

Morris, Jane. 2006. Readers' Perceptions of Lexical Cohesion and Lexical Semantic Relations in Text. Ph.D. thesis, University of Toronto.

[36]

Morris, Jane and Graeme Hirst. 1991. Lexical cohesion computed by thesaural relations as an indicator of the structure of text. Computational Linguistics, 17(1):21-48.

[37]

Morris, Jane and Graeme Hirst. 2004. Non-classical lexical semantic relations. In Workshop on Computational Lexical Semantics, Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, Boston, MA, pages 46-51, May.

[38]

Morris, Jane and Graeme Hirst. 2005. The subjectivity of lexical cohesion in text. In James G. Shanahan, Yan Qu, and Janyce Wiebe, editors, Computing Attitude and Affect in Text. Springer, New York, pages 41-48.

[39]

Murphy, M. Lynne. 2003. Semantic Relations and the Lexicon. Cambridge University Press.

[40]

Patwardhan, Siddharth, Satanjeev Banerjee, and Ted Pedersen. 2003. Using measures of semantic relatedness for word sense disambiguation. In Proceedings of the Fourth International Conference on Intelligent Text Processing and Computational Linguistics, Mexico City, Mexico, pages 241-257, February.

[41]

Pedersen, Ted, Siddharth Patwardhan, and Jason Michelizzi. 2004. Wordnet:: similarity--measuring the relatedness of concepts. In Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI-04). AAAI Press, Cambridge, MA, pages 1024-1025.

[42]

Procter, Paul, editor. 1978. Longman Dictionary of Contemporary English. Longman.

[43]

Rada, Roy and Ellen Bicknell. 1989. Ranking documents with a thesaurus. JASIS, 40(5):304-310.

[44]

Rada, Roy, Hafedh Mili, Ellen Bicknell, and Maria Blettner. 1989. Development and application of a metric on semantic nets. IEEE Transactions on Systems, Man, and Cybernetics, 19(1):17-30.

[45]

Resnik, Philip. 1995. Using information content to evaluate semantic similarity. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448-453, Montreal, Canada, August.

[46]

Resnik, Philip and Mona Diab. 2000. Measuring verb similarity. In Proceedings of the 22nd Annual Meeting of the Cognitive Science Society, Philadelphia, PA, pages 399-404.

[47]

Rubenstein, Herbert and John B. Goodenough. 1965. Contextual correlates of synonymy. Communications of the ACM, 8(10):627-633.

[48]

Spellman, Barbara A., Keith J. Holyoak, and Robert G. Morrison. 2001. Analogical priming via semantic relations. Memory and Cognition, 29(3):383-393.

[49]

St-Onge, David. 1995. Detecting and correcting malapropisms with lexical chains. Master's thesis, University of Toronto. Published as Technical Report CSRI-319.

[50]

Stevenson, Mark and Mark Greenwood. 2005. A semantic approach to IE pattern induction. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05), pages 379-386, Ann Arbor, MI, June.

[51]

Sussna, Michael. 1993. Word sense disambiguation for free-text indexing using a massive semantic network. In Proceedings of the Second International Conference on Information and Knowledge Management (CIKM-93), pages 67-74, Arlington, VA.

[52]

Sussna, Michael John. 1997. Text Retrieval Using Inference in Semantic Metanetworks. Ph.D. thesis, University of California, San Diego.

[53]

Weeds, Julie and David Weir. 2005. Co-occurrence retrieval: A flexible framework for lexical distributional similarity. Computational Linguistics, 31(4):433-475.

[54]

Weeds, Julie E. 2003. Measures and applications of lexical distributional similarity. Ph.D. thesis, University of Sussex, September.

[55]

Wei, Mei. 1993. An analysis of word relatedness correlation measures. Master's thesis, University of Western Ontario, London, Ontario.

[56]

Whitten, William B., W. Newton Suter, and Michael L. Frank. 1979. Bidirectional synonym ratings of 464 noun pairs. Journal of Learning and Verbal Behavior, 18:109-127.

[57]

Wu, Zhibiao and Martha Palmer. 1994. Verb semantics and lexical selection. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, pages 133-138, Las Cruces, NM, June.

Cited By

Zhang DChatzimparmpas AKamali NHullman J(2024)Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image LabelingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642446(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642446
Baloi ABelean BTurcu FPeptenatu D(2024)GPU-based similarity metrics computation and machine learning approaches for string similarity evaluation in large datasetsSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-023-08687-828:4(3465-3477)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s00500-023-08687-8
Lovón-Melgarejo JMoreno JBesançon RFerret OTamine L(2024)Probing Pretrained Language Models with Hierarchy PropertiesAdvances in Information Retrieval10.1007/978-3-031-56060-6_9(126-142)Online publication date: 24-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56060-6_9
Show More Cited By

Index Terms

Evaluating WordNet-based Measures of Lexical Semantic Relatedness

Recommendations

Semantic disambiguation of wordnet glosses and lexical chains on extended wordnet
WordNet-based semantic relatedness measures in automatic speech recognition for meetings
ACL '07: Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions

This paper presents the application of WordNet-based semantic relatedness measures to Automatic Speech Recognition (ASR) in multi-party meetings. Different word-utterance context relatedness measures and utterance-coherence measures are defined and ...
Building Synsets for Indonesian WordNet with Monolingual Lexical Resources
IALP '10: Proceedings of the 2010 International Conference on Asian Language Processing

This paper presents an approach to build synsets for Indonesian Word Net semi-automatically using monolingual lexical resources available freely in Bahasa Indonesia. Monolingual lexical resources refer to Kamus Besar Bahasa Indoensia or KBBI (...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computational Linguistics

Computational Linguistics Volume 32, Issue 1

March 2006

158 pages

ISSN:0891-2017

EISSN:1530-9312

Issue’s Table of Contents

Publisher

MIT Press

Cambridge, MA, United States

Publication History

Published: 01 March 2006

Published in COLI Volume 32, Issue 1

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

372
Total Citations
View Citations
2,880
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang DChatzimparmpas AKamali NHullman J(2024)Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image LabelingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642446(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642446
Baloi ABelean BTurcu FPeptenatu D(2024)GPU-based similarity metrics computation and machine learning approaches for string similarity evaluation in large datasetsSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-023-08687-828:4(3465-3477)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s00500-023-08687-8
Lovón-Melgarejo JMoreno JBesançon RFerret OTamine L(2024)Probing Pretrained Language Models with Hierarchy PropertiesAdvances in Information Retrieval10.1007/978-3-031-56060-6_9(126-142)Online publication date: 24-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56060-6_9
Hafeez HMuneer ISharjeel MAshraf MAdeel Nawab R(2023)Urdu Short Paraphrase Detection at Sentence LevelACM Transactions on Asian and Low-Resource Language Information Processing10.1145/358600922:4(1-20)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1145/3586009
Mehak GMuneer INawab R(2023)Urdu Text Reuse Detection at Phrasal level using Sentence Transformer-based approachExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.121063234:COnline publication date: 30-Dec-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.121063
González Parets OSimón-Cuevas AOlivas JPerea-Ortega J(2023)A Hybrid Approach for Spanish Emotion Recognition Applying Fuzzy Semantic ProcessingProgress in Artificial Intelligence and Pattern Recognition10.1007/978-3-031-49552-6_20(225-236)Online publication date: 3-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-49552-6_20
Mehler ABagci MHenlein AAbrami GSpiekermann CSchrottenbacher PKonca MLücking AEngel JQuintino MSchreiber JSaukel KZlatkin-Troitschanskaia O(2023)A Multimodal Data Model for Simulation-Based Learning with Va.Si.Li-LabDigital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management10.1007/978-3-031-35741-1_39(539-565)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-35741-1_39
Bitton YBitton-Guetta NYosef RElovici YBansal MStanovsky GSchwartz RKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)WinoGAViLProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3602195(26549-26564)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3602195
Martinez-Rodriguez JLopez-Arevalo IRios-Alvarado A(2022)Mining information from sentences through Semantic Web data and Information Extraction tasksJournal of Information Science10.1177/016555152093438748:1(3-20)Online publication date: 1-Feb-2022
https://dl.acm.org/doi/10.1177/0165551520934387
Segarra-Faggioni VRatté SDe Jong F(2022)Semantic similarity analysis between the student's reports with the authoritative sourcesProceedings of the 14th International Conference on Education Technology and Computers10.1145/3572549.3572645(599-605)Online publication date: 28-Oct-2022
https://dl.acm.org/doi/10.1145/3572549.3572645
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents