Buckle: Evaluating fact checking algorithms built on knowledge bases

VP Huynh, P Papotti - Proceedings of the VLDB Endowment, 2019 - dl.acm.org
Proceedings of the VLDB Endowment, 2019dl.acm.org
Fact checking is the task of determining if a given claim holds. Several algorithms have been
developed to check facts with reference information in the form of knowledge bases. We
demonstrate BUCKLE, an open-source benchmark for comparing and evaluating fact
checking algorithms in a level playing field across a range of scenarios. The demo is
centered around three main lessons. To start, we show how, by changing the properties of
the training and test facts, it is possible to influence significantly the performance of the …
Fact checking is the task of determining if a given claim holds. Several algorithms have been developed to check facts with reference information in the form of knowledge bases. We demonstrate BUCKLE, an open-source benchmark for comparing and evaluating fact checking algorithms in a level playing field across a range of scenarios. The demo is centered around three main lessons. To start, we show how, by changing the properties of the training and test facts, it is possible to influence significantly the performance of the algorithms. We then show the role of the reference data. Finally, we discuss the performance for algorithms designed on different principles and assumptions, as well as approaches that address the link prediction task in knowledge bases.
ACM Digital Library