Hostname: page-component-cd9895bd7-jn8rn Total loading time: 0 Render date: 2024-12-14T00:12:07.391Z Has data issue: false hasContentIssue false

When the answer comes into question in question-answering: survey and open issues

Published online by Cambridge University Press:  10 January 2012

ANA CRISTINA MENDES
Affiliation:
Spoken Language Systems Laboratory - L2F/INESC-ID Instituto Superior Técnico, Technical University of Lisbon, R. Alves Redol, 9 - 2° – 1000-029 Lisboa, Portugal e-mail: ana.mendes@l2f.inesc-id.pt, luisa.coheur@l2f.inesc-id.pt
LUÍSA COHEUR
Affiliation:
Spoken Language Systems Laboratory - L2F/INESC-ID Instituto Superior Técnico, Technical University of Lisbon, R. Alves Redol, 9 - 2° – 1000-029 Lisboa, Portugal e-mail: ana.mendes@l2f.inesc-id.pt, luisa.coheur@l2f.inesc-id.pt

Abstract

The answer determines the success of a Question-Answering (QA) system. In redundancy-based QA systems, a common approach is to extract the candidate answers from the information sources and select the most frequent answers as the final answers. However, this strategy has some pitfalls. For instance, if a system is not able to detect equivalences between the candidate answers, their frequencies might be erroneously calculated. Moreover, the user who posed the question should also be taken into account when answering: different persons require different (correct) answers. This can involve the use of suitable vocabulary and/or information details. In these situations, the generation of a response can be a more suitable strategy, instead of the extraction and direct retrieval of the answer from the information sources. The present survey targets the state of the art in the answering task in QA under three different lines of research. First, we present several works that focus on relating candidate answers. Then, we recover the concept of cooperative answer – a correct, useful, and non-misleading answer – and we bring up attempts to address cooperative answering. Finally, we investigate the research community endeavors on response generation. We will also present our perspective on each of these three topics throughout this paper.

Type
Articles
Copyright
Copyright © Cambridge University Press 2012

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Androutsopoulos, I., Ritchie, G. D. and Thanisch, P. 1995. Natural language interfaces to databases—an introduction. Natural Language Engineering 1: 2981.CrossRefGoogle Scholar
Balahur, A., Boldrini, E., Montoyo, A. and Martínez-Barco, P. 2010. Going beyond traditional QA systems: challenges and keys in opinion question answering. In Proceedings of the 23rd International Conference on Computational Linguistics COLING '10: Posters, Beijing, China, pp. 27–35. Stroudsburg, PA: ACL.Google Scholar
Benamara, F. 2004a. Generating intensional answers in intelligent question answering systems. In Belz, A.et al. (eds.), International Conference on Natural Language Generation (INLG), pp. 1120. Lecture Notes in Artificial Intelligence. Berlin: Springer-Verlag.CrossRefGoogle Scholar
Benamara, F. 2004b. Cooperative question answering in restricted domains: the WEBCOOP experiment. In Mollá, D. and Vicedo, J. L. (eds.), ACL 2004: Question Answering in Restricted Domains, pp. 31–8. Stroudsburg, PA: ACL.Google Scholar
Benamara, F. and Saint-Dizier, P. 2003. Dynamic generation of cooperative natural language responses. In Proceedings of the 9th European Workshop on Natural Language Generation (ENLG-2003), Budapest, Hungary, pp. 5667. Stroudsburg, PA: ACL.Google Scholar
Benamara, F. and Saint-Dizier, P. 2004. Lexicalisation strategies in cooperative question answering systems. In Gambäck, B. and Jokinen, K. (eds.), Proceedings of the 20th International Conference on Computational Linguistics, (COLING '04) pp. 1179–85. Stroudsburg, PA: ACL.CrossRefGoogle Scholar
Blair-Goldensohn, S., Mckeown, K. R., and Schlaikjer, A. H. 2003. A hybrid approach for QA track definitional questions. In Proceedings of the 12th Annual Text Retrieval Conference (TREC 2003), Gaithersburg, MD, USA, pp. 185–92.Google Scholar
Bos, J., Curran, J. R. and Guzzetti, E. 2007. The pronto QA system at TREC-2007: harvesting hyponyms, using nominalisation patterns, and computing answer cardinality. In Voorhees, E. M. and Buckland, L. P. (eds.), Proceedings of the 16th Text RETrieval Conference, TREC 2007, Gaithersburg, MD, USA, pp. 726–32.Google Scholar
Bosma, W. E. 2005. Extending answers using discourse structure. In Saggion, H. and Minel, J. L. (eds.), RANLP Workshop on Crossing Barriers in Text Summarization Research, Borovets, Bulgaria, pp. 29.Google Scholar
Breck, E. J., Burger, J. D., Ferro, L., Hirschman, L., House, D., Light, M., and Mani, I. 2000. How to evaluate your question answering system every day and still get real work done. In Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC-2000), Athens, Greece, pp. 1495–500.Google Scholar
Breck, E. J., Burger, J., Ferro, L., House, D., Light, M., and Mani, I. 1999. A sys called qanda. In Voorhees, E. M. and Harman, D. K. (eds.), Proceedings of the 8th Text REtrieval Conference (TREC 8), Gaithersburg, MD, pp. 499506.Google Scholar
Brill, E., Dumais, S. and Banko, M. 2002. An analysis of the AskMSR question-answering system. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing – Volume 10, pp. 257–64. Stroudsburg, PA: ACL.Google Scholar
Brill, E., Lin, J., Banko, M., Dumais, S., and Ng, A. 2001. Data-intensive question answering. In Proceedings of the 10th Text REtrieval Conference (TREC 2001), ithersburg, MD, pp. 393400.Google Scholar
Buchholz, S. and Daelemans, W. 2001. Complex answers: a case study using a WWW question answering system. Natural Language Engineering 7: 301–23.CrossRefGoogle Scholar
Cardie, C., Ng, V., Pierce, D. and Buckley, C. 2000. Examining the role of statistical and linguistic knowledge sources in a general-knowledge question-answering system. In Proceedings of the 6th Conference on Applied Natural Language Processing, pp. 180–87. Stroudsburg, PA: ACL.CrossRefGoogle Scholar
Chklovski, T. and Pantel, P. 2004. VerbOcean: mining the web] for fine-grained semantic verb relations. In Lin, D. and Wu, D. (eds.), Proceedings of the EMNLP 2004, pp. 3340. Stroudsburg, PA: ACL.Google Scholar
Clarke, C. L. A., Cormack, G. V. and Lynam, T. R. 2001. Exploiting redundancy in question answering. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM.Google Scholar
Curtis, J., Matthews, G. and Baxter, D. 2005. On the effective use of cyc in a question answering system. In Proceedings of the IJCAI Workshop on Knowledge and Reasoning for Answering Questions, Edinburgh, Scotland, pp. 6170.Google Scholar
Dalmas, T. and Webber, B. 2005. Using information fusion for open domain question answering. Proceedings of the KRAQ Workshop, IJCAI, Edinburgh, Scotland.Google Scholar
Dalmas, T. and Webber, B. 2007. Answer comparison in automated question answering. Journal of Applied Logic 5: 104–20.CrossRefGoogle Scholar
Dang, H. T., Kelly, D. and Lin, J. J. 2007. Overview of the TREC 2007 question answering track. In Voorhees, E. M. and Buckland, L. P. (eds.), Proceedings of The Sixteenth Text REtrieval Conference, TREC 2007, Gaithersburg, MD, pp. 137–45. Gaithersburg, MD: NIST.Google Scholar
De Boni, M., and Manandhar, S. 2003. An analysis of clarification dialogue for question answering. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology – Volume 1, pp. 4855. Stroudsburg, PA: ACL.Google Scholar
De Marneffe, M., Rafferty, A. N., and Manning, C. D. 2008. Finding contradictions in text. In Proceedings of ACL-08: HLT, pp. 1039–47. Stroudsburg, PA: ACL.Google Scholar
Demner-Fushman, D., and Lin, J. 2006. Answer extraction, semantic clustering, and extractive summarization for clinical question answering. In ACL-44: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pp. 841–8. Stroudsburg, PA: ACL.Google Scholar
Fellbaum, C. 1998. WordNet: An Electronic Lexical Database. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Ferrucci, D., Nyberg, E., Allan, J., Barker, K., Brown, E., Chu-Carroll, J., Ciccolo, A., Duboue, P., Fan, J., Gondek, D., Hovy, E., Katz, B., Lally, A., McCord, M., Morarescu, P., Murdock, B., Porter, B., Prager, J., Welty, C., and Zadrozny, W. 2009. Towards the open advancement of question answering systems. IBM Technical Report, RC 24789, IBM Research Division, New York.Google Scholar
Forner, P., Peñas, A., Agirre, E., Alegria, I., Forăscu, C., Moreau, N., Osenova, P., Prokopidis, P., Rocha, P., Sacaleanu, B., Sutcliffe, R., and Sang, E. T. K. 2009. Overview of the Clef 2008 multilingual question answering track. In Proceedings of the 9th Cross-language Evaluation Forum Conference on Evaluating Systems for Multilingual and Multimodal Information Access, pp. 262–95. Berlin: Springer-Verlag.CrossRefGoogle Scholar
Gaasterland, T., Godfrey, P. and Minker, J. 1992. An overview of cooperative answering. Journal of Intelligent Information Systems 1: 123–57.CrossRefGoogle Scholar
Grice, H. 1975. Logic and Conversation. Syntax and Semantics, vol. 3, Speech Acts. Maryland Heights, MO: Academic Press.Google Scholar
Harabagiu, S. and Maiorano, S. 1999. Finding answers in large collections of texts: paragraph indexing + abductive inference. In Proceedings of the AAAI Fall Symposium on Question Answering Systems, Toulouse, France, pp. 6371.Google Scholar
Harabagiu, S., Hickl, A. and Lacatusu, F. 2006. Negation, contrast and contradiction in text processing. In AAAI'06: Proceedings of the 21st National Conference on Artificial Intelligence, pp. 755–62. Palo Alto, CA: AAAI Press.Google Scholar
Harabagiu, S., Moldovan, D., Paşca, M., Mihalcea, R., Surdeanu, M., Bunescu, R., Girju, R., Rus, V., Morerescu, P. 2000. FALCON – boosting knowledge for answer engines. Proceedings of Text REtrieval Conference (TREC-9), Gaithersburg, MD.Google Scholar
Hearst, M. A. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of the 14th Conference on Computational Linguistics, pp. 539–45. Stroudsburg, PA: ACL.CrossRefGoogle Scholar
Hermjakob, U. 2001. Parsing and question classification for question answering. In Proceedings of the Workshop on Open-Domain Question Answering – Volume 12, pp. 16. Stroudsburg, PA: ACL.Google Scholar
Huang, Z., Thint, M. and Qin, Z. 2008. Question classification using head words and their hypernyms. In EMNLP '08: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 927–36. Stroudsburg, PA: ACL.CrossRefGoogle Scholar
Hung, J. C., Wang, C., Yang, C., Chiu, M., and Yee, G. 2005. Applying word sense disambiguation to question answering system for e-learning. In AINA '05: Proceedings of the 19th International Conference on Advanced Information Networking and Applications, pp. 157–62. Washington, DC: IEEE Computer Society.Google Scholar
Iftene, A. and Balahur, A. 2008. Answer validation on English and Romanian languages. In Peters, C.et al. (eds.), Evaluating Systems for Multilingual and Multimodal Information Access, pp. 448–51. Lecture Notes in Computer Science. Berlin: Springer-Verlag.Google Scholar
Ittycheriah, A., Franz, M. and Roukos, S. 2001. IBM's statistical question answering system – TREC-10. Proceedings of the 10th Text REtrieval Conference (TREC-10), Gaithersburg, MD.Google Scholar
Jurafsky, D. and Martin, J. H. 2008. Speech and Language Processing, 2nd ed. (Prentice Hall Series in Artificial Intelligence). New Jersey: Prentice Hall.Google Scholar
Kamp, H. and Reyle, U. 1993. From Discourse to Logic: Introduction to Model-Theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Netherlands: Kluwer.Google Scholar
Kaplan, S. 1979. Cooperative Responses from a Portable Natural Language Data Base Query System. PhD thesis, University of Pennsylvania.Google Scholar
Kaplan, S. 1981. Appropriate responses to inappropriate questions. In Elements of Discourse Understanding, pp. 127144. Cambridge, UK: Cambridge University Press.Google Scholar
Kehler, A. 2002. Coherence, Reference and the Theory of Grammar. Stanford, CA: CSLI.Google Scholar
Ko, J., Si, L., and Nyberg, E. 2007. A probabilistic framework for answer selection in question answering. In Sidner, C. L.et al. (eds.), Proceedings of Human Language Technology Conference of the NAACL, pp. 524–31. Stroudsburg, PA: ACL.Google Scholar
Kozareva, Z., Vazquez, S. and Montoyo, A. 2006. Adaptation of a machine-learning textual entailment system to a multilingual answer validation exercise. In Working Notes of CLEF–ECDL 2006, AVE Workshop, LNCS 3237. Berlin: Springer-Verlag.Google Scholar
Kwok, C., Etzioni, O. and Weld, D. S. 2001. Scaling question answering to the web. ACM Transactions on Information Systems 19: 242–62.CrossRefGoogle Scholar
Leung, Y. K. and Apperley, M. D. 1994. A review and taxonomy of distortion-oriented presentation techniques. ACM Transactions on Computer-Human Interaction 1: 126–60.Google Scholar
Lin, J. 2007. An exploration of the principles underlying redundancy-based factoid question answering. ACM Transactions on Information Systems 25: 155.CrossRefGoogle Scholar
Lin, J. and Katz, B. 2006. Building a reusable test collection for question answering. Journal of the American Society for Information Science and Technology 57: 851–61.CrossRefGoogle Scholar
Lin, J., Quan, D., Sinha, V., Bakshi, K., Huynh, D., Katz, B., and Karger, D. R. 2003. What makes a good answer? The role of context in question answering. In Rauterberg, M.et al. (eds.) Human-Computer Interaction (INTERACT 2003), Zurich, Switzerland, pp. 2532.Google Scholar
Liu, F., Yu, C. and Meng, W. 2004. Personalized web search for improving retrieval effectiveness. IEEE Transactions on Knowledge and Data Engineering 16: 2840.Google Scholar
Lin, D., Zhao, S., Qin, L. and Zhou, M. 2002. Learning question classifiers. In Proceedings of the 19th International Conference on Computational Linguistics, pp. 17. Stroudsburg, PA: ACL.Google Scholar
Magnini, B., Negri, M., Prevete, R. and Tanev, H. 2002. Is it the right answer?: exploiting web redundancy for answer validation. In ACL '02: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 425–32. Stroudsburg, PA: ACL.Google Scholar
Magnini, B., Romagnoli, S., Vallin, A., Herrera, J., Peñas, A., Peinado, V., Verdejo, F., De Rijke, M., and Vallin, R. 2003. The multiple language question answering track at CLEF 2003. In CLEF 2003. CLEF 2003 Workshop. Berlin: Springer-Verlag.Google Scholar
Magnini, B., Vallin, R., Ayache, C., Erbach, G., Peñas, A., De Rijke, M., Rocha, P., Kiril, S., and Sutcliffe, R. 2005. Overview of the CLEF 2004 multilingual question answering track. In Results of the CLEF 2004 Cross-Language System Evaluation Campaign, pp. 371–91. Berlin: Springer-Verlag.Google Scholar
Mann, W. C. and Thompson, S. A. 1988. Rhetorical structure theory: toward a functional theory of text organization. Text 8: 243–81.Google Scholar
Mendes, A. C. and Coheur, L. 2011. An approach to answer selection in question-answering based on semantic relations. In Walsh, T. (ed.), IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, pp. 1852–1857.Google Scholar
Mendes, A. C., Prada, R. and Coheur, L. 2009. Adapting a virtual agent to users' vocabulary and needs. In Rutkay, et al. (eds.), IVA '09: Proceedings of the 9th International Conference on Intelligent Virtual Agents, pp. 529–30. Lecture Notes in Artificial Intelligence. Berlin: Springer-Verlag.Google Scholar
Moldovan, D., Clark, C., Harabagiu, S. and Maiorano, S. 2003. COGEX: a logic prover for question answering. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology – Volume 1, pp. 8793. Stroudsburg, PA: ACL.Google Scholar
Moldovan, D., Harabagiu, S., Paşca, M., Mihalcea, R., Girju, R., Goodrum, R., and Rus, V. 2001. The structure and performance of an open-domain question answering system. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pp. 563–70. Stroudsburg, PA: ACL.Google Scholar
Moldovan, D., Paşca, M., Harabagiu, S. and Surdeanu, M. 2003. Performance issues and error analysis in an open-domain question answering system. ACM Transactions on Information Systems 21: 133–54.Google Scholar
Mollá, D. and Gardiner, M. 2004. Answerfinder: question answering by combining lexical, syntactic and semantic information. In Proceedings of the Australasian Language Technology Workshop 2004, NSW, Australia, pp. 916.Google Scholar
Moriceau, V. 2005. Answer generation with temporal data integration. In Wilcock, G.et al. (eds.), Proceedings of the 10th European Workshop on Natural Language Generation (ENLG-05), University of Aberdeen, Aberdeen, UK, pp. 197202.Google Scholar
Moriceau, V. 2006. Numerical data integration for cooperative question-answering. In Proceedings of the Workshop KRAQ'06 on Knowledge and Reasoning for Language Processing, pp. 4350. Stroudsburg, PA: ACL.Google Scholar
Moriceau, V. 2007) Intégration de Données dans un Système Question-Réponse sur le Web. PhD thesis, Université Paul Sabatier, Toulouse, France.Google Scholar
Nii, Y., Kawata, K., Yoshida, T., Sakai, H., and Masuyama, S. 2004. Question answering system QUARK. Proceedings of the NTCIR-4, Tokyo, Japan.Google Scholar
Nyberg, E., Mitamura, T., Callan, J. P., Carbonell, J. G., Frederking, R., Collins-Thompson, K., Hiyakumoto, L., Huang, Y., Huttenhower, C., Judy, S., Ko, J., Kupsc, A., Lita, L. V., Pedro, V., Svoboda, D., and Van Durme, B. 2003. The JAVELIN question-answering system at TREC 2003: a multi-strategy approach with dynamic planning. In Proceedings of the 2003 Text REtrieval Conference (TREC 2003), Gaithersburg, MD.Google Scholar
Nyberg, E., Mitamura, T., Carbonell, J. G., Callan, J. P., Collins-Thompson, K., Czuba, K., Duggan, M., Hiyakumoto, L., Hu, N., Huang, Y., Ko, J., Lita, L. V., Murtagh, S., Pedro, V., and Svoboda, D. 2002. The JAVELIN question-answering system at TREC 2002. In Proceedings of the TREC'02, Gaithersburg, MD.Google Scholar
Pantel, P. and Ravichandran, D. 2004. Automatically labeling semantic classes. In Dumais, S., Marcu, D., and Roukos, S. (eds.) Proceedings of the North American Association for Computational Linguistics and the Human Language Technologies Conferences (NAACL/HLT2004), pp. 321–28. Stroudsburg, PA: ACL.Google Scholar
Paşca, M. and Harabagiu, S. 2001. The informative role of WordNet in open-domain question answering. In Proceedings of the 2nd Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL-01), Workshop on WordNet and Other Lexical Resources: Applications, Extensions and Customizations. Stroudsburg, PA: ACL.Google Scholar
Quarteroni, S. and Manandhar, S. 2007. User modelling for personalized question answering. In Basili, R. and Pazienza, M. (eds.), AI*IA '07: Proceedings of the 10th Congress of the Italian Association for Artificial Intelligence on AI*IA 2007, pp. 386–97. Lecture Notes in Computer Science. Berlin: Springer-Verlag.Google Scholar
Quarteroni, S. and Manandhar, S. 2008. Designing an interactive open-domain question answering system. Journal of Natural Language Engineering 15: 7395.CrossRefGoogle Scholar
Ritter, A., Soderland, S. and Etzioni, O. 2009. What is this, anyway: automatic hypernym discovery. In Proceedings of AAAI-09 Spring Symposium on Learning by Reading and Learning to Read, pp. 8893. Palo Alto, CA: AAAI.Google Scholar
Rodrigo, Á., Peñas, A., and Verdejo, F. 2009. Overview of the answer validation exercise 2008. In Belz, A.et al. (eds.), Proceedings of the 9th Cross-Language Evaluation Forum Conference on Evaluating Systems for Multilingual and Multimodal Information Access, pp. 296313. Berlin: Springer-Verlag.CrossRefGoogle Scholar
Rosset, S., Galibert, O., Illouz, G. and Max, A. 2006. Integrating Spoken Dialog and Question Answering: The Ritel Project. In Proceedings of Interspeech 2006—ICSLP: 9th International Conference on Spoken Language Processing, Pittsburgh, PA, pp. 1914–17.Google Scholar
Schlobach, S., Ahn, D., de Rijke, M., and Jijkoun, V. 2007. Data-driven type checking in open domain question answering. Journal of Applied Logic 5: 121–43.CrossRefGoogle Scholar
Shanon, B. 1979. Where questions. In Proceedings of the 17th Annual Meeting on Association for Computational Linguistics, pp. 73–5. Stroudsburg, PA: ACL.CrossRefGoogle Scholar
Silva, J., Coheur, L., Mendes, A. and Wichert, A. 2011. From symbolic to sub-symbolic information in question classification. Artificial Intelligence Review 35: 137–54.CrossRefGoogle Scholar
Small, S. and Strazalkowski, T. 2009. HITIQA: high-quality intelligence through interactive question answering. Natural Language Engineering 15: 3154.CrossRefGoogle Scholar
Teevan, J., Dumais, S. T. and Horvitz, E. 2005. Personalizing search via automated analysis of interests and activities. In SIGIR '05: Proceedings of the 28th Annual Int. ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 449–56. New York: ACM.CrossRefGoogle Scholar
Téllez-Valero, A., Juárez-González, A., Montes-y-Gómez, M., and Villaseñor-Pineda, L. 2008. INAOE at QA@CLEF 2008: evaluating answer validation in Spanish question answering. In Working Notes of CLEF–ECDL 2008, AVE Workshop, Aarhus, Denmark. Berlin: Springer-Verlag.Google Scholar
Téllez-Valero, A., Montes-Gómez, M., Villaseñor-Pineda, L., and Peñas, A. 2010. Towards multi-stream question answering using answer validation. Informatica. Special Issue on Computational Linguistics and its Applications 34: 4554.Google Scholar
Vargas-Vera, M., and Motta, E. 2004. AQUA” ontology-based question answering system. In MICAI 2004: Advances in Artificial Intelligence, pp. 468–77. Berlin: Springer.Google Scholar
Verberne, S. 2009. In Search of the Why: Developing a System for Answering Why-Questions. PhD thesis, University of Nijmegen, Nijmegen.Google Scholar
Voorhees, E. M. and Tice, D. M. 1999. The TREC-8 question answering track evaluation. In In Text Retrieval Conference TREC-8, Gaithersburg, MD, pp. 83105.CrossRefGoogle Scholar
Webber, B., Gardent, C. and Bos, J. 2002. Position statement: inference in question answering. In Proceedings of the LREC Workshop on Question Answering: Strategy and Resources, Canary Islands, Spain, pp. 1925.Google Scholar
Wu, H., Radev, D. R. and Fan, W. 2002. Towards answer focused summarization. Proceedings of the 1st International Conference on Information Technology and Applications.Google Scholar
Ye, S., Chua, T. and Lu, J. 2009. Summarizing definition from Wikipedia. In ACL-IJCNLP '09: Proceedings of the Joint Conf. of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1, pp. 199207. Stroudsburg, PA: ACL.Google Scholar