Abstract
Question answering systems aim to meet users’ information needs by returning exact answers in response to a question. Traditional open domain question answering systems are built around a single pipeline architecture. In an attempt to exploit multiple resources as well as multiple answering strategies, systems based on a multi-stream architecture have recently been introduced. Such systems face the challenging problem of having to select a single answer from pools of answers obtained using essentially different techniques. We report on experiments aimed at understanding and evaluating the effect of different options for answer selection in a multi-stream question answering system. We examine the impact of local tiling techniques, assignments of weights to streams based on past performance and/or question type, as well redundancy-based ideas. Our main finding is that redundancy-based ideas in combination with naively learned stream weights conditioned on question type work best, and improve significantly over a number of baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Chu-Carrol, J., Prager, J., Welty, C., Czuba, K., Ferrucci, D.: A multi-strategy and multi-source approach to question answering. In: Proceedings of TREC 2002, pp. 281–288 (2003)
Clarke, C.L.A., Cormack, G.V., Kemkes, G., Laszlo, M., Lynam, T.R., Terra, E.L., Tilker, P.L.: Statistical selection of exact answers. In: Proceedings of TREC 2002, pp. 823–831 (2003)
Croft, W.B.: Combining approaches to information retrieval. In: Croft, W.B. (ed.) Advances in Information Retrieval, pp. 1–36. Kluwer Academic Publishers, Dordrecht (2000)
de Chalendar, G., Dalmas, T., Elkateb-Gara, F., Ferret, O., Hurault-Plantet, M., Grau, B., Illouz, G., Monceaux, L., Robba, I., Vilnat, A.: The Question Answering System QALC at LIMSI, Experiments in Using Web and WordNet. In: Proceedings of TREC 2002, pp. 407–416 (2003)
Banko, M., et al.: AskMSR: Question answering using the Worldwide Web. In: Proc. EMNLP 2002 (2002)
Fleischman, M., Hovy, E.H., Echihabi, A.: Offline strategies for online question answering: Answering questions before they are asked. In: Proceedings ACL 2003, pp. 1–7 (2003)
Fox, E.A., Shaw, J.A.: Combination of multiple searches. In: Proceedings TREC-2, pp. 243–252 (1994)
Greenwood, M., Roberts, I., Gaizauskas, R.: The University of Sheffield TREC 2002 Q&A system. In: Proceedings of TREC 2002, pp. 823–831 (2003)
Hiemstra, D., Kraaij, W., Pohlmann, R., Westerveld, T.: Translation resources, merging strategies, and relevance feedback for cross-language information retrieval. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 102–115. Springer, Heidelberg (2001)
Hovy, E., Hermjakob, H., Junk, M., Lin, C.-Y.: Question answering in Webclopedia. In: Proceedings TREC-9 (2000)
Jijkoun, V., Kamps, J., Mishne, G., Monz, C., de Rijke, M., Schlobach, S., Tsur, O.: The University of Amsterdam at TREC 2003. In: TREC 2003 Notebook Papers (2003)
Jijkoun, V., Mishne, G., de Rijke, M.: How frogs built the Berlin Wall. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 523–534. Springer, Heidelberg (2004) (to appear)
Magnini, B., Negri, M., Prevete, R., Tanev, H.: Is it the right answer? Exploiting web redundancy for answer validation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 425–432 (2002)
Moldovan, D., Harabagiu, S., Paşca, M., Mihalcea, R., Girju, R., Goodrum, R., Rus, V.: The structure and performance of an open domain question answering system. In: Proceedings ACL 2000, pp. 563–570 (2000)
Moldovan, D., Paşca, M., Harabagiu, S., Surdeanu, M.: Performance issues and error analysis in an open-domain question answering system. ACM Transactions on Information Systems 21, 133–154 (2003)
Monz, C., de Rijke, M.: Tequesta: The University of Amsterdam’s textual question answering system. In: Proceedings TREC 2001, pp. 519–528 (2002)
Prager, J., Brown, E., Coden, A., Radev, D.: Question-answering by predicitive annotation. In: Proceedings SIGIR 2000, pp. 184–191 (2000)
Soubbotin, M.M., Soubbotin, S.M.: Use of patterns for detection of likely answer strings: A systematic approach. In: Proceedings TREC 2002, pp. 325–331 (2003)
Text REtreival Conference (TREC), http://trec.nist.gov
Xu, J., Licuanan, A., May, J., Miller, S., Weischedel, R.: TREC 2002 QA at BBN: Answer Selection and Confidence Estimation. In: Proceedings of TREC 2002, pp. 96–101 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jijkoun, V., de Rijke, M. (2004). Answer Selection in a Multi-stream Open Domain Question Answering System. In: McDonald, S., Tait, J. (eds) Advances in Information Retrieval. ECIR 2004. Lecture Notes in Computer Science, vol 2997. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24752-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-540-24752-4_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21382-6
Online ISBN: 978-3-540-24752-4
eBook Packages: Springer Book Archive