Issue Downloads
Evaluation-as-a-Service for the Computational Sciences: Overview and Outlook
- Frank Hopfgartner,
- Allan Hanbury,
- Henning Müller,
- Ivan Eggel,
- Krisztian Balog,
- Torben Brodt,
- Gordon V. Cormack,
- Jimmy Lin,
- Jayashree Kalpathy-Cramer,
- Noriko Kando,
- Makoto P. Kato,
- Anastasia Krithara,
- Tim Gollub,
- Martin Potthast,
- Evelyne Viegas,
- Simon Mercer
Evaluation in empirical computer science is essential to show progress and assess technologies developed. Several research domains such as information retrieval have long relied on systematic evaluation to measure progress: here, the Cranfield paradigm ...
Anserini: Reproducible Ranking Baselines Using Lucene
This work tackles the perennial problem of reproducible baselines in information retrieval research, focusing on bag-of-words ranking models. Although academic information retrieval researchers have a long history of building and sharing systems, they ...
Reproducible Web Corpora: Interactive Archiving with Automatic Quality Assessment
The evolution of web pages from static HTML pages toward dynamic pieces of software has rendered archiving them increasingly difficult. Nevertheless, an accurate, reproducible web archive is a necessity to ensure the reproducibility of web-based ...
To Clean or Not to Clean: Document Preprocessing and Reproducibility
Web document collections such as WT10G, GOV2, and ClueWeb are widely used for text retrieval experiments. Documents in these collections contain a fair amount of non-content-related markup in the form of tags, hyperlinks, and so on. Published articles ...