More Web Proxy on the site http://driver.im/

research-article

Complex Factoid Question Answering with a Free-Text Knowledge Graph

Authors:

Jordan Boyd-GraberAuthors Info & Claims

WWW '20: Proceedings of The Web Conference 2020

Pages 1205 - 1216

https://doi.org/10.1145/3366423.3380197

Published: 20 April 2020 Publication History

Abstract

We introduce delft, a factoid question answering system which combines the nuance and depth of knowledge graph question answering approaches with the broader coverage of free-text. delft builds a free-text knowledge graph from Wikipedia, with entities as nodes and sentences in which entities co-occur as edges. For each question, delft finds the subgraph linking question entity nodes to candidates using text sentences as edges, creating a dense and high coverage semantic graph. A novel graph neural network reasons over the free-text graph—combining evidence on the nodes via information along edge sentences—to select a final answer. Experiments on three question answering datasets show delft can answer entity-rich questions better than machine reading based models, bert-based answer ranking and memory networks. delft’s advantage comes from both the high coverage of its free-text knowledge graph—more than double that of dbpedia relations—and the novel graph neural network which reasons on the rich but noisy free-text evidence.

References

[1]

Junwei Bao, Nan Duan, Ming Zhou, and Tiejun Zhao. 2014. Knowledge-based Question Answering as Machine Tsranslation. In Proceedings of the Association for Computational Linguistics.

[2]

Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic Parsing on Freebase from Question-Answer Pairs. In Proceedings of Empirical Methods in Natural Language Processing.

[3]

Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A Collaboratively Created Graph Database for Structuring Human knowledge. In Proceedings of the ACM SIGMOD international conference on Management of data.

Digital Library

[4]

Antoine Bordes, Sumit Chopra, and Jason Weston. 2014. Question Answering with Subgraph Embeddings. In Proceedings of Empirical Methods in Natural Language Processing.

[5]

Antoine Bordes, Nicolas Usunier, Sumit Chopra, and Jason Weston. 2015. Large-scale Simple Question Answering with Memory Networks. arXiv preprint arXiv:1506.02075(2015).

[6]

Jordan Boyd-Graber, Brianna Satinoff, He He, and Hal Daume III. 2012. Besting the Quiz Master: Crowdsourcing Incremental Classification Games. In Proceedings of Empirical Methods in Natural Language Processing.

Digital Library

[7]

Qingqing Cai and Alexander Yates. 2013. Large-scale Semantic Parsing via Schema Matching and Lexicon Extension. In Proceedings of the Association for Computational Linguistics.

[8]

Jamie Callan, Mark Hoy, Changkuk Yoo, and Le Zhao. 2009. Clueweb09 data set.

[9]

Jonathan Chang, Jordan Boyd-Graber, and David M. Blei. 2009. Connections between the Lines: Augmenting Social Networks with Text. In Knowledge Discovery and Data Mining.

[10]

Danqi Chen. 2018. Neural Reading Comprehension and Beyond. Ph.D. Dissertation. Stanford University.

[11]

Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to Answer Open-Domain Questions. In Proceedings of the Association for Computational Linguistics.

[12]

Christopher Clark and Matt Gardner. 2018. Simple and Effective Multi-Paragraph Reading Comprehension. In Proceedings of the Association for Computational Linguistics.

[13]

Nicola De Cao, Wilker Aziz, and Ivan Titov. 2019. Question Answering by Reasoning Across Documents with Graph Convolutional Networks. In Conference of the North American Chapter of the Association for Computational Linguistics.

[14]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Conference of the North American Chapter of the Association for Computational Linguistics.

[15]

Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang. 2019. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. In Proceedings of the Association for Computational Linguistics. Proceedings of the Association for Computational Linguistics.

[16]

Li Dong, Furu Wei, Ming Zhou, and Ke Xu. 2015. Question Answering over Freebase with Multi-Column Convolutional Neural Networks. In Proceedings of the Association for Computational Linguistics.

[17]

Xin Luna Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. 2014. Knowledge Vault: A Web-Scale Approach to Probabilistic Knowledge Fusion. In Knowledge Discovery and Data Mining.

Digital Library

[18]

Mohnish Dubey, Debayan Banerjee, Abdelrahman Abdelkawi, and Jens Lehmann. 2019. Lc-quad 2.0: A Large Dataset for Complex Question Answering over Wikidata and Dbpedia. In International Semantic Web Conference.

[19]

Ahmed Elgohary, Chen Zhao, and Jordan Boyd-Graber. 2018. Dataset and Baselines for Sequential Open-Domain Question Answering. In Proceedings of Empirical Methods in Natural Language Processing.

[20]

Paolo Ferragina and Ugo Scaiella. 2010. Tagme: On-the-fly Annotation of Short Text Fragments (by Wikipedia Entities). In Proceedings of the ACM International Conference on Information and Knowledge Management.

Digital Library

[21]

David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A. Kalyanpur, Adam Lally, J. William Murdock, Eric Nyberg, John Prager, Nico Schlaefer, and Chris Welty. 2010. Building Watson: An Overview of the DeepQA Project. AI Magazine 31, 3 (2010).

[22]

Matt Gardner, Jonathan Berant, Hannaneh Hajishirzi, Alon Talmor, and Sewon Min. 2019. Question Answering is a Format; When is it Useful?arXiv preprint arXiv:1909.11291(2019).

[23]

Matt Gardner and Jayant Krishnamurthy. 2017. Open-Vocabulary Semantic Parsing with Both Distributional Statistics and Formal Knowledge. In Association for the Advancement of Artificial Intelligence.

[24]

Clinton Gormley and Zachary Tong. 2015. Elasticsearch: The Definitive Guide(1st ed.). O’Reilly Media, Inc.

Digital Library

[25]

Mohit Iyyer, Jordan Boyd-Graber, Leonardo Claudino, Richard Socher, and Hal Daumé III. 2014. A Neural Network for Factoid Question Answering over Paragraphs. In Proceedings of Empirical Methods in Natural Language Processing.

[26]

Mohit Iyyer, Anupam Guha, Snigdha Chaturvedi, Jordan Boyd-Graber, and Hal Daumé III. 2016. Feuding Families and Former Friends: Unsupervised Learning for Dynamic Fictional Relationships. In North American Association for Computational Linguistics.

[27]

Ken Jennings. 2006. Brainiac: adventures in the curious, competitive, compulsive world of trivia buffs. Villard.

[28]

Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension. In Proceedings of the Association for Computational Linguistics.

[29]

Thomas N Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of the International Conference on Learning Representations.

[30]

Tom Kwiatkowski, Eunsol Choi, Yoav Artzi, and Luke Zettlemoyer. 2013. Scaling Semantic Parsers with On-the-fly Ontology Matching. In Proceedings of Empirical Methods in Natural Language Processing.

[31]

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, 2019. Natural Questions: A Benchmark for Question Answering Research. In Transactions of the Association for Computational Linguistics.

[32]

Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent Retrieval for Weakly Supervised Open Domain Question Answering. In Proceedings of the Association for Computational Linguistics.

[33]

Xiaolu Lu, Soumajit Pramanik, Rishiraj Saha Roy, Abdalghani Abujabal, Yafang Wang, and Gerhard Weikum. 2019. Answering Complex Questions by Joining Multi-Document Evidence with Quasi Knowledge Graphs. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.

Digital Library

[34]

Pablo N Mendes, Max Jakob, Andrés García-Silva, and Christian Bizer. 2011. DBpedia Spotlight: Shedding Light on the Web of Documents. In Proceedings of the International Conference on Semantic Systems.

Digital Library

[35]

Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-Value Memory Networks for Directly Reading Documents. In Proceedings of Empirical Methods in Natural Language Processing. Proceedings of the Association for Computational Linguistics.

[36]

Sewon Min, Eric Wallace, Sameer Singh, Matt Gardner, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019. Compositional Questions Do Not Necessitate Multi-hop Reasoning. In Proceedings of the Association for Computational Linguistics.

[37]

Sewon Min, Victor Zhong, Richard Socher, and Caiming Xiong. 2018. Efficient and Robust Question Answering from Minimal Context over Documents. In Proceedings of the Association for Computational Linguistics.

[38]

Sewon Min, Victor Zhong, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2019. Multi-hop Reading Comprehension through Question Decomposition and Rescoring. In Proceedings of the Association for Computational Linguistics.

[39]

Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms MARCO: A Human Generated Machine Reading Comprehension Dataset. arXiv preprint arXiv:1611.09268(2016).

[40]

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic Differentiation in PyTorch. (2017).

[41]

Heiko Paulheim. 2018. How much is a Triple? Estimating the Cost of Knowledge Graph Creation. In International Semantic Web Conference.

[42]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global Vectors for Word Representation. In Proceedings of Empirical Methods in Natural Language Processing.

[43]

Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, and Yong Yu. 2019. Dynamically Fused Graph Network for Multi-hop Reasoning. In Proceedings of the Association for Computational Linguistics.

[44]

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Proceedings of Empirical Methods in Natural Language Processing.

[45]

Siva Reddy, Mirella Lapata, and Mark Steedman. 2014. Large-Scale Semantic Parsing without Question-Answer Pairs. In Transactions of the Association for Computational Linguistics.

[46]

Gerard Salton and Michael J McGill. 1983. Introduction to Modern Information Retrieval. mcgraw-hill.

[47]

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2009. The Graph Neural Network Model. In IEEE Transactions on Neural Networks.

[48]

Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2017. Modeling Relational Data with Graph Convolutional Networks. arXiv preprint arXiv:1703.06103(2017).

[49]

Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional Attention Flow for Machine Comprehension. In Proceedings of the International Conference on Learning Representations.

[50]

Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: A Core of Semantic Knowledge. In Proceedings of the World Wide Web Conference.

Digital Library

[51]

Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William Cohen. 2018. Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text. In Proceedings of Empirical Methods in Natural Language Processing.

[52]

Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xiaodong He, and Bowen Zhou. 2019. Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs. In Proceedings of the Association for Computational Linguistics.

[53]

Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: A Free Collaborative Knowledge Base. (2014).

[54]

Mengqiu Wang. 2006. A Survey of Answer Extraction Techniques in Factoid Question Answering. Computational Linguistics1 (2006).

[55]

Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018. Constructing Datasets for Multi-Hop Reading Comprehension Across Documents. Transactions of the Association for Computational Linguistics.

[56]

Jason Weston, Sumit Chopra, and Antoine Bordes. 2014. Memory Networks. arXiv preprint arXiv:1410.3916(2014).

[57]

Chenyan Xiong. 2018. Text Representation, Retrieval, and Understanding with Knowledge Graphs. Ph.D. Dissertation. Carnegie Mellon University.

[58]

Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In Proceedings of Advances in Neural Information Processing Systems.

[59]

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In Proceedings of Empirical Methods in Natural Language Processing.

[60]

Xuchen Yao and Benjamin Van Durme. 2014. Information Extraction Over Structured Data: Question Answering with Freebase. In Proceedings of the Association for Computational Linguistics.

[61]

Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2015. Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base. In Proceedings of the Association for Computational Linguistics.

[62]

Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V Le. 2018. QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension. In Proceedings of the International Conference on Learning Representations.

[63]

Victor Zhong, Caiming Xiong, Nitish Shirish Keskar, and Richard Socher. 2019. Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering. In Proceedings of the International Conference on Learning Representations.

Cited By

Yoon SKo SKim TKang SYeo JLee DNejdl WAuer SKarras OCha MMoens MNajork M(2025)Unsupervised Robust Cross-Lingual Entity Alignment via Neighbor Triple Matching with Entity and Relation TextsProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703500(184-193)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703500
Jafarzadeh PEnsan FAli Akbar Alavi MZarrinkalam F(2025)A Knowledge Graph Embedding Model for Answering Factoid Entity QuestionsACM Transactions on Information Systems10.1145/367800343:2(1-27)Online publication date: 24-Jan-2025
https://dl.acm.org/doi/10.1145/3678003
Andreasen TBordogna GTré GKacprzyk JLarsen HZadrożny S(2024)The power and potentials of Flexible Query Answering SystemsData & Knowledge Engineering10.1016/j.datak.2023.102246149:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.datak.2023.102246
Show More Cited By

Index Terms

Complex Factoid Question Answering with a Free-Text Knowledge Graph

Index terms have been assigned to the content through auto-classification.

Recommendations

Knowledge Graph Embedding Based Question Answering
WSDM '19: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining

Question answering over knowledge graph (QA-KG) aims to use facts in the knowledge graph (KG) to answer natural language questions. It helps end users more efficiently and more easily access the substantial and valuable knowledge in the KG, without ...
A Knowledge Graph Embedding Model for Answering Factoid Entity Questions
Factoid entity questions (FEQ), which seek answers in the form of a single entity from knowledge sources, such as DBpedia and Wikidata, constitute a substantial portion of user queries in search engines. This article introduces the knowledge graph ...
Knowledge Graph Question Answering with Ambiguous Query
WWW '23: Proceedings of the ACM Web Conference 2023

Knowledge graph question answering aims to identify answers of the query according to the facts in the knowledge graph. In the vast majority of the existing works, the input queries are considered perfect and can precisely express the user’s query ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '20: Proceedings of The Web Conference 2020

April 2020

3143 pages

ISBN:9781450370233

DOI:10.1145/3366423

Editors:
Yennun Huang
Acadmica sinica, Taiwan
,
Irwin King
The Chinese University of Hong Kong, Hong Kong
,
Tie-Yan Liu
Microsoft Research Asia, China
,
Maarten van Steen
University of Twente, Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '20

Sponsor:

SIGWEB

WWW '20: The Web Conference 2020

April 20 - 24, 2020

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
1,587
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yoon SKo SKim TKang SYeo JLee DNejdl WAuer SKarras OCha MMoens MNajork M(2025)Unsupervised Robust Cross-Lingual Entity Alignment via Neighbor Triple Matching with Entity and Relation TextsProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703500(184-193)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703500
Jafarzadeh PEnsan FAli Akbar Alavi MZarrinkalam F(2025)A Knowledge Graph Embedding Model for Answering Factoid Entity QuestionsACM Transactions on Information Systems10.1145/367800343:2(1-27)Online publication date: 24-Jan-2025
https://dl.acm.org/doi/10.1145/3678003
Andreasen TBordogna GTré GKacprzyk JLarsen HZadrożny S(2024)The power and potentials of Flexible Query Answering SystemsData & Knowledge Engineering10.1016/j.datak.2023.102246149:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.datak.2023.102246
Jafarzadeh PAmirmahani ZEnsan F(2024)Learning contextual representations for entity retrievalApplied Intelligence10.1007/s10489-024-05430-054:19(8820-8840)Online publication date: 4-Jul-2024
https://doi.org/10.1007/s10489-024-05430-0
Jafarzadeh PEnsan F(2024)An evidence-based approach for open-domain question answeringKnowledge and Information Systems10.1007/s10115-024-02269-267:2(1969-1991)Online publication date: 15-Nov-2024
https://doi.org/10.1007/s10115-024-02269-2
Yang JYang LWang HGao YZhao YXie XLu Y(2023)Representation learning for knowledge fusion and reasoning in Cyber–Physical–Social Systems: Survey and perspectivesInformation Fusion10.1016/j.inffus.2022.09.00390(59-73)Online publication date: Feb-2023
https://doi.org/10.1016/j.inffus.2022.09.003
Suissa OZhitomirsky-Geffet MElmalech A(2022)Question answering with deep neural networks for semi-structured heterogeneous genealogical knowledge graphsSemantic Web10.3233/SW-22292514:2(209-237)Online publication date: 15-Dec-2022
https://doi.org/10.3233/SW-222925
Jafarzadeh PAmirmahani ZEnsan FAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)Learning to Rank Knowledge Subgraph Nodes for Entity RetrievalProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531888(2519-2523)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531888
Ye ZKumar YSing GSong FWang J(2022)A Comprehensive Survey of Graph Neural Networks for Knowledge GraphsIEEE Access10.1109/ACCESS.2022.319178410(75729-75741)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3191784
Bi XNie HZhang XZhao XYuan YWang G(2022)Unrestricted multi-hop reasoning network for interpretable question answering over knowledge graphKnowledge-Based Systems10.1016/j.knosys.2022.108515243:COnline publication date: 11-May-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.108515
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten