More Web Proxy on the site http://driver.im/

article

Are you sure that this happened? assessing the factuality degree of events in text

Authors:

James PustejovskyAuthors Info & Claims

Computational Linguistics, Volume 38, Issue 2

Pages 261 - 299

https://doi.org/10.1162/COLI_a_00096

Published: 01 June 2012 Publication History

Abstract

Identifying the veracity, or factuality, of event mentions in text is fundamental for reasoning about eventualities in discourse. Inferences derived from events judged as not having happened, or as being only possible, are different from those derived from events evaluated as factual. Event factuality involves two separate levels of information. On the one hand, it deals with polarity, which distinguishes between positive and negative instantiations of events. On the other, it has to do with degrees of certainty (e.g., possible, probable), an information level generally subsumed under the category of epistemic modality. This article aims at contributing to a better understanding of how event factuality is articulated in natural language. For that purpose, we put forward a linguistic-oriented computational model which has at its core an algorithm articulating the effect of factuality relations across levels of syntactic embedding. As a proof of concept, this model has been implemented in De Facto, a factuality profiler for eventualities mentioned in text, and tested against a corpus built specifically for the task, yielding an F₁ of 0.70 (macro-averaging) and 0.80 (micro-averaging). These two measures mutually compensate for an over-emphasis present in the other (either on the lesser or greater populated categories), and can therefore be interpreted as the lower and upper bounds of the De Facto's performance.

References

[1]

ACE, 2008. Automatic Content Extraction. English Annotation Guidelines for Relations. Linguistic Data Consortium, version 6.0- 2008.01.07 edition. Available at http://www.ldc.upenn.edu/Projects/ACE/.

[2]

Aikhenvald, Alexandra Y. 2004. Evidentiality. Oxford University Press, Oxford.

[3]

Asher, Nicholas. 1993. Reference to Abstract Objects in English. Kluwer Academic Press, Dordrecht.

[4]

Bach, Kent and Robert M. Harnish. 1979. Linguistic Communication and Speech Acts. The MIT Press, Cambridge, MA.

[5]

Biber, Douglas and Edward Finegan. 1989. Styles of stance in English: Lexical and grammatical marking of evidentiality and affect. Text, 9(1):93-124.

[6]

Chafe, Wallace. 1986. Evidentiality in English conversation and academic writing. In W. Chafe and J. Nichols, editors, Evidentiality: The Linguistic Coding of Epistemology. Ablex Publishing Corporation, Norwood, NJ.

[7]

Dalianis, Hercules and Maria Skeppstedt. 2010. Creating and evaluating a consensus for negated and speculative words in a Swedish clinical corpus. In Proceedings of the Workshop on Negation and Speculation in Natural Language Processing, pages 5-13, Uppsala, Sweden.

[8]

de Haan, Ferdinand. 1997. The Interaction of Modality and Negation: a Typological Study. Garland, New York.

[9]

de Marneffe, Marie-Catherine, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of LREC 2006, pages 449-454, Genoa, Italy.

[10]

Diab, Mona, Bonnie Dorr, Lori Levin, Teruko Mitamura, Rebecca Passonneau, Owen Rambow, and Lance Ramshaw. 2009a. Language Understanding Annotation Corpus. Linguistic Data Consortium, Philadelphia, PA. LDC2009T10.

[11]

Diab, Mona T., Lori Levin, Teruko Mitamura, Owen Rambow, Vinodkumar Prabhakaran, and Weiwei Guo. 2009b. Committed belief annotation and tagging. In Proceedings of the Third Linguistic Annotation Workshop, ACL-IJNLP'09, pages 68-73, Suntec, Singapore.

[12]

Dor, Daniel. 1995. Representations, Attitudes and Factivity Evaluations. An Epistemically-based Analysis of Lexical Selection. Ph.D. thesis, Stanford University.

[13]

Farkas, Richárd, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010. The CoNLL-2010 shared task: Learning to detect hedges and their scope in natural language text. In Proceedings of the 14th CoNLL Conference - Shared Task, pages 1-12, Uppsala, Sweden.

[14]

Geurts, Bart. 1998. Presuppositions and anaphors in attitude contexts. Linguistics and Philosophy, 21:545-601.

[15]

Givón, Talmy. 1993. English Grammar. A Function-Based Introduction. John Benjamins, Amsterdam.

[16]

Glanzberg, Michael. 2003. Felicity and presupposition triggers. In University of Michigan Workshop in Philosophy and Linguistics, Michigan.

[17]

Halliday, M. A. K. and Christian M. I. M. Matthiessen. 2004. An Introduction to Functional Grammar. Hodder Arnold, London.

[18]

Henriksson, Aron and Sumithra Velupillai. 2010. Levels of certainty in knowledge-intensive corpora: An initial annotation study. In Proceedings of the Workshop on Negation and Speculation in Natural Language Processing, pages 41-45, Uppsala, Sweden.

[19]

Hickl, Andrew and Jeremy Bensley. 2007. A discourse commitment-based framework for recognizing textual entailment. In Proceedings of the Workshop on Textual Entailment and Paraphrasing, pages 171-176, Prague.

[20]

Hooper, Joan B. 1975. On assertive predicates. In J. Kimball, editor, Syntax and Semantics, IV. Academic Press, New York, pages 91-124.

[21]

Horn, Laurence R. 1972. On the Semantic Properties of Logical Operators in English. Ph.D. thesis, UCLA. Distributed by the Indiana University Linguistics Club, 1976.

[22]

Horn, Laurence R. 1989. A Natural History of Negation. University of Chicago Press, Chicago, IL.

[23]

Hyland, Ken. 1996. Writing without conviction? Hedging in science research articles. Applied Linguistics, 14(4):433-454.

[24]

Karttunen, Lauri. 1970. Implicative verbs. Language, 47:340-358.

[25]

Karttunen, Lauri and Annie Zaenen. 2005. Veridicity. In G. Katz, J. Pustejovsky, and F. Schilder, editors, Dagstuhl Seminar Proceedings, Schloss Dagstuhl, link: http://drops.dagstuhl.de/opus/ volltexte/2005/314/pdf/05151. KarttunenLauri.Paper.314.pdf.

[26]

Kiefer, Ferenc. 1987. On defining modality. Folia Linguistica, XXI:67-94.

[27]

Kilicoglu, Halil and Sabine Bergler. 2008. Recognizing speculative language in biomedical research articles: A linguistically motivated perspective. BMC Bioinformatics, 9(Suppl 11):S10.

[28]

Kim, Jin-Dong, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun'ichi Tsujii. 2009. Overview of BioNLP'09 shared task on event extraction. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing: Shared Task, pages 1-9. Boulder, Colorado, USA.

[29]

Kim, Jin-Dong, Tomoko Ohta, and Jun'ichi Tsujii. 2008. Corpus annotation for mining biomedical events from literature. BMC Bioinformatics, 9(1):10.

[30]

Kiparsky, Paul and Carol Kiparsky. 1970. Fact. In M. Bierwisch and K. E. Heidolph, editors, Progress in Linguistics. A Collection of Papers. Mouton, The Hague, Paris, pages 143-173.

[31]

Kratzer, Angelika. 1991. Modality. In A. van Stechow and D. Wunderlich, editors, Semantik: Ein internationales Handbuch der zeitgenoessischen Forschung. Walter de Gruyter, Berlin, pages 639-650.

[32]

Kudo, Taku and Yuji Matsumoto. 2000. Use of support vector learning for chunk identification. In Proceedings of CoNLL-2000 and LLL-2000, pages 142-144, Lisbon, Portugal.

[33]

Lakoff, George. 1973. Hedges: A study in meaning criteria and the logic of fuzzy concepts. Journal of Philosophical Logic, 2(4):458-508.

[34]

Light, Marc, Xin Ying Qiu, and Padmini Srinivasan. 2004. The language of bioscience: Facts, speculations, and statements in between. In BioLINK 2004: Linking Biological Literature, Ontologies, and Databases, pages 17-24, Boston, Massachusetts, USA.

[35]

Lyons, John. 1977. Semantics. Cambridge University Press, Cambridge.

[36]

Martin, James R. and Peter R. R. White. 2005. Language of Evaluation: Appraisal in English. London and New York: Palgrave Macmillan.

[37]

Medlock, Ben and Ted Briscoe. 2007. Weakly supervised learning for hedge classification in scientific literature. In Proceedings of the 45th ACL, pages 992-999, Prague, Czech Republic.

[38]

Moilanen, Karo and Stephen Pulman. 2007. Sentiment composition. In Proceedings of the RANLP, pages 27-29, Borovets, Bulgaria.

[39]

Moilanen, Karo, Stephen Pulman, and Yue Zhang. 2010. Packed feelings and ordered sentiments: Sentiment parsing with quasi-compositional polarity sequencing and compression. In Proceedings of the 1st Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2010), pages 36-43, Alacant, Spain.

[40]

Morante, Roser and Walter Daelemans. 2009a. Learning the scope of hedge cues in biomedical texts. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing, pages 28-36, Boulder, Colorado, USA.

[41]

Morante, Roser and Walter Daelemans. 2009b. A metalearning approach to processing the scope of negation. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning, pages 21-29, Boulder, Colorado, USA.

[42]

Mushin, Ilana. 2001. Evidentiality and Epistemological Stance. John Benjamin, Philadelphia, PA.

[43]

Nairn, Rowan, Cleo Condoravdi, and Lauri Karttunen. 2006. Computing relative polarity for textual inference. In Inference in Computational Semantics, ICoS-5, pages 67-76, Buxton, England.

[44]

Nawaz, Raheel, Paul Thompson, and Sophia Ananiadou. 2010. Evaluating a meta-knowledge annotation scheme for bio-events. In Proceedings of the Workshop on Negation and Speculation in Natural Language Processing, pages 69-77, Uppsala, Sweden.

[45]

Neviarouskaya, Alena, Helmut Prendinger, and Mitsuru Ishizuka. 2009. Compositionality principle in recognition of fine-grained emotions from text. In Proceedings of the 3rd International ICWSM Conference, pages 278-281, San Jose, California, USA.

[46]

Ohta, Tomoko, Jin-Dong Kim, and Jun'ichi Tsuji. 2007. Guidelines for event annotation. University of Tokyo, link: http://www-tsujii.is.s.u-tokyo.ac.jp/~genia/release/Genia_event_annotation_guidelines.pdf.

[47]

Özgür, Arzucan and Dragomir Radev. 2009. Detecting speculations and their scopes in scientific text. In Proceedings of the 2009 EMNLP Conference, pages 1398-1407, Suntec, Singapore.

[48]

Palmer, Frank R. 1986. Mood and Modality. Cambridge University Press, Cambridge.

[49]

Polanyi, Livia and Annie Zaenen. 2006. Contextual valence shifters. In W. B. Croft, J. Shanahan, Y. Qu, and J. Wiebe, editors, Computing Attitude and Affect in Text: Theory and Applications, volume 20. Springer-Verlag, New York, pages 1-10.

[50]

Prabhakaran, Vinodkumar, Owen Rambow, and Mona Diab. 2010. Automatic committed belief tagging. In CoLing 2010. Poster Volume, pages 1014-1022, Beijing, China.

[51]

Prasad, Rashmi, Nikhil Dinesh, Alan Lee, Aravind Joshi, and Bonnie Webber. 2007. Attribution and its annotation in the Penn Discourse Treebank. Traitement Automatique des Langues, 47(2):43-64.

[52]

Pustejovsky, James, Bob Knippen, Jessica Littman, and Roser Saurí. 2005. Temporal and event information in natural language text. Language Resources and Evaluation, 39(2):123-164.

[53]

Pustejovsky, James, Marc Verhagen, Roser Saurí, Jessica Littman, Robert Gaizauskas, Graham Katz, Inderjeet Mani, Robert Knippen, and Andrea Setzer. 2006. TimeBank 1.2. Linguistic Data Consortium, Philadelphia, PA. LDC2006T08.

[54]

Rizomilioti, Vassiliki. 2006. Exploring epistemic modality in academic discourse using corpora. In E. Arnó Macià, A. Soler Cervera, and C. Rueda Ramos, editors, Information Technology in Languages for Specific Purposes, volume 7. Springer, Berlin, pages 53-71.

[55]

Rubin, Victoria L. 2006. Identifying Certainty in Texts. Ph.D. thesis, Syracuse University.

[56]

Rubin, Victoria L. 2007. Stating with certainty or stating with doubt: Intercoder reliability results for manual annotation of epistemically modalized statements. In Proceedings of the NAACL-HLT 2007, pages 141-144, Rochester, NY.

[57]

Rubin, Victoria L. 2010. Epistemic modality: From uncertainty to certainty in the context of information seeking as interactions with texts. Information Processing and Management, 46:533-540.

[58]

Saurí, Roser. 2008. A Factuality Profiler for Eventualities in Text. Ph.D. thesis, Brandeis University.

[59]

Saurí, Roser and James Pustejovsky. 2007. Determining modality and factuality for text entailment. In Proceedings of the First IEEE International Conference on Semantic Computing, pages 509-516, Irvine, California, USA.

[60]

Saurí, Roser and James Pustejovsky. 2009a. FactBank 1.0. Linguistic Data Consortium, Philadelphia, PA. LDC2009T23.

[61]

Saurí, Roser and James Pustejovsky. 2009b. FactBank. A corpus annotated with event factuality. Language Resources and Evaluation, 43:227-268.

[62]

Saurí, Roser, Marc Verhagen, and James Pustejovsky. 2006. SlinkET: A partial modal parser for events. In Proceedings of LREC 2006, pages 1332-1337, Genoa, Italy.

[63]

Shatkay, Hagit, Fengxia Pang, Andrey Rzhetsky, and W. John Wilbur. 2008. Multi-dimensional classification of biomedical text: Toward automated, practical provision of high-utility text to diverse users. Bioinformatics, 24:2086-2093.

[64]

Szarvas, Gyorgy. 2008. Hedge classification in biomedical texts with a weakly supervised selection of keywords. In ACL 08: HLT, pages 281-289, Columbus, Ohio, USA.

[65]

van Valin, Robert D. and Randy J. LaPolla. 1997. Syntax. Structure, Meaning and Function. Cambridge University Press, Cambridge.

[66]

Velldal, Erik, Lilja Ovrelid, and Stephan Oepen. 2010. Resolving speculation: Maxent cue classification and dependency-based scope rules. In Proceedings of the 14th CoNLL: Shared Task, pages 48-55, Uppsala, Sweden.

[67]

Vincze, Veronika, György Szarvas, Richárd Farkas, György Móra, and János Csirik. 2008. The BioScope corpus: Biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 9(Suppl 11):S9.

[68]

Wiebe, Janyce, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation, 39(2):165-210.

[69]

Wilbur, W. John, Andrey Rzhetsky, and Hagit Shatkay. 2006. New directions in biomedical text annotation: definitions, guidelines and corpus construction. BMC Bioinformatics, 7(1):356-365.

Cited By

Zymla MBuchmüller RButt MKeim D(2024)Deciphering Personal Argument Styles – A Comprehensive Approach to Analyzing Linguistic Properties of Argument PreferencesRobust Argumentation Machines10.1007/978-3-031-63536-6_18(296-314)Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-63536-6_18
Zhang ZQian ZZhu XLi P(2023)CoDE: Contrastive Learning Method for Document-Level Event Factuality IdentificationDatabase Systems for Advanced Applications10.1007/978-3-031-30675-4_36(497-512)Online publication date: 17-Apr-2023
https://dl.acm.org/doi/10.1007/978-3-031-30675-4_36
Liu XHuang HZhang Y(2022)End-to-end event factuality prediction using directional labeled graph recurrent networkInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10283659:2Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.ipm.2021.102836
Show More Cited By

Are you sure that this happened? assessing the factuality degree of events in text

Recommendations

Exploring events and distributed representations of text in multi-document summarization

We explore an event detection framework to improve multi-document summarizationWe use distributed representations of text to address different lexical realizationsSummarization is based on the hierarchical combination of single-document summariesWe ...
A factuality profiler for eventualities in text
Event Factuality Detection in Discourse
Natural Language Processing and Chinese Computing
Abstract
Event factuality indicates whether an event occurs or the degree of certainty described by authors in context. Correctly identifying event factuality in texts can contribute to a deep understanding of natural language. In addition, event ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computational Linguistics

Computational Linguistics Volume 38, Issue 2

June 2012

230 pages

ISSN:0891-2017

EISSN:1530-9312

Issue’s Table of Contents

Publisher

MIT Press

Cambridge, MA, United States

Publication History

Published: 01 June 2012

Published in COLI Volume 38, Issue 2

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
442
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zymla MBuchmüller RButt MKeim D(2024)Deciphering Personal Argument Styles – A Comprehensive Approach to Analyzing Linguistic Properties of Argument PreferencesRobust Argumentation Machines10.1007/978-3-031-63536-6_18(296-314)Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-63536-6_18
Zhang ZQian ZZhu XLi P(2023)CoDE: Contrastive Learning Method for Document-Level Event Factuality IdentificationDatabase Systems for Advanced Applications10.1007/978-3-031-30675-4_36(497-512)Online publication date: 17-Apr-2023
https://dl.acm.org/doi/10.1007/978-3-031-30675-4_36
Liu XHuang HZhang Y(2022)End-to-end event factuality prediction using directional labeled graph recurrent networkInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10283659:2Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.ipm.2021.102836
Zhang ZLiu CQian ZZhu XLi P(2022)HSN: Heterogeneous Semantics-Syntax Fusion Network for Document-Level Event Factuality IdentificationPRICAI 2022: Trends in Artificial Intelligence10.1007/978-3-031-20865-2_23(309-320)Online publication date: 10-Nov-2022
https://dl.acm.org/doi/10.1007/978-3-031-20865-2_23
Kouadri WAkoka JComyn-Wattiau Idu Mouza C(2022)Uncertainty Detection in Historical DatabasesNatural Language Processing and Information Systems10.1007/978-3-031-08473-7_7(73-85)Online publication date: 15-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-08473-7_7
Sinha MAgarwal NDasgupta THuang RWu DMarchionini GHe DCunningham SHansen P(2020)Relation Aware Attention Model for Uncertainty Detection in TextProceedings of the ACM/IEEE Joint Conference on Digital Libraries in 202010.1145/3383583.3398613(437-440)Online publication date: 1-Aug-2020
https://dl.acm.org/doi/10.1145/3383583.3398613
Qian ZLi PZhang YZhou GZhu Q(2018)Event factuality identification via generative adversarial networks with auxiliary classificationProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304222.3304367(4293-4300)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304222.3304367
Fuhr NGiachanou AGrefenstette GGurevych IHanselowski AJarvelin KJones RLiu YMothe JNejdl WPeters IStein B(2018)An Information Nutritional Label for Online DocumentsACM SIGIR Forum10.1145/3190580.319058851:3(46-66)Online publication date: 22-Feb-2018
https://dl.acm.org/doi/10.1145/3190580.3190588
Qian ZLi PZhou GZhu Q(2018)Event Factuality Identification via Hybrid Neural NetworksNeural Information Processing10.1007/978-3-030-04221-9_30(335-347)Online publication date: 13-Dec-2018
https://dl.acm.org/doi/10.1007/978-3-030-04221-9_30
Sahu IMajumdar D(2017)Detecting Factual and Non-Factual Content in News ArticlesProceedings of the 4th ACM IKDD Conferences on Data Sciences10.1145/3041823.3041837(1-12)Online publication date: 9-Mar-2017
https://dl.acm.org/doi/10.1145/3041823.3041837
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents