More Web Proxy on the site http://driver.im/

research-article

To Clean or Not to Clean: Document Preprocessing and Reproducibility

Authors:

Debasis GangulyAuthors Info & Claims

Journal of Data and Information Quality (JDIQ), Volume 10, Issue 4

Article No.: 18, Pages 1 - 25

https://doi.org/10.1145/3242180

Published: 29 October 2018 Publication History

Abstract

Web document collections such as WT10G, GOV2, and ClueWeb are widely used for text retrieval experiments. Documents in these collections contain a fair amount of non-content-related markup in the form of tags, hyperlinks, and so on. Published articles that use these corpora generally do not provide specific details about how this markup information is handled during indexing. However, this question turns out to be important: Through experiments, we find that including or excluding metadata in the index can produce significantly different results with standard IR models. More importantly, the effect varies across models and collections. For example, metadata filtering is found to be generally beneficial when using BM25, or language modeling with Dirichlet smoothing, but can significantly reduce retrieval effectiveness if language modeling is used with Jelinek-Mercer smoothing. We also observe that, in general, the performance differences become more noticeable as the amount of metadata in the test collections increase. Given this variability, we believe that the details of document preprocessing are significant from the point of view of reproducibility. In a second set of experiments, we also study the effect of preprocessing on query expansion using RM3. In this case, once again, we find that it is generally better to remove markup before using documents for query expansion.

References

[1]

Giambattista Amati. 2003. Probability Models for Information Retrieval Based on Divergence from Randomness. Ph.D. Dissertation. University of Glasgow.

[2]

Gianni Amati and Cornelis Joost Van Rijsbergen. 2002. Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans. Inf. Syst. 20, 4 (Oct. 2002), 357--389.

Digital Library

[3]

Yael Anava, Anna Shtok, Oren Kurland, and Ella Rabinovich. 2016. A probabilistic fusion framework. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management (CIKM’16). ACM, New York, NY, 1463--1472.

Digital Library

[4]

Jaime Arguello, Matt Crane, Fernando Diaz, Jimmy Lin, and Andrew Trotman. 2016. Report on the SIGIR 2015 workshop on reproducibility, inexplicability, and generalizability of results (RIGOR). ACM SIGIR Forum 49, 2 (2016), 107--116.

Digital Library

[5]

Timothy G. Armstrong, Alistair Moffat, William Webber, and Justin Zobel. 2009. Improvements that don’t add up: Ad-hoc retrieval results since 1998. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, (CIKM’09), David Wai-Lok Cheung, Il-Yeol Song, Wesley W. Chu, Xiaohua Hu, and Jimmy J. Lin (Eds.). ACM, 601--610.

Digital Library

[6]

Hosein Azarbonyad, Mostafa Dehghani, Maarten Marx, and Jaap Kamps. 2015. Time-aware authorship attribution for short text streams. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). ACM, New York, NY, 727--730.

Digital Library

[7]

Leif Azzopardi, Yashar Moshfeghi, Martin Halvey, Rami S. Alkhawaldeh, Krisztian Balog, Emanuele Di Buccio, Diego Ceccarelli, Juan M. Fernández-Luna, Charlie Hull, Jake Mannix, and Sauparna Palchowdhury. 2017. Lucene4IR: Developing information retrieval evaluation resources using Lucene. SIGIR Forum 50, 2 (Feb. 2017), 58--75.

Digital Library

[8]

Peter Bailey, Nick Craswell, and David Hawking. 2003. Engineering a multi-purpose test collection for web retrieval experiments. Inf. Process. Manage. 39, 6 (Nov. 2003), 853--871.

Digital Library

[9]

Saeid Balaneshin-kordan and Alexander Kotov. 2016. Sequential query expansion using concept graph. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management (CIKM’16). ACM, New York, NY, 155--164.

Digital Library

[10]

Stefan Büttcher, Charles L. A. Clarke, and Ian Soboroff. {n.d.}. The TREC 2006 terabyte track. Retrieved from https://trec.nist.gov/pubs/trec15/papers/TERA06.OVERVIEW.pdf.

[11]

Deng Cai, Shipeng Yu, Ji-Rong Wen, and Wei-Ying Ma. 2003. Extracting content structure for web pages based on visual representation. In Proceedings of the 5th Asia-Pacific Web Conference on Web Technologies and Applications (APWeb’03). Springer-Verlag, Berlin, 406--417.

Digital Library

[12]

Carlos Castillo and Brian D. Davison. 2011. Adversarial web search. Found. Trends Inf. Retr. 4, 5 (May 2011), 377--486.

Digital Library

[13]

Carlos Castillo, Debora Donato, Luca Becchetti, Paolo Boldi, Stefano Leonardi, Massimo Santini, and Sebastiano Vigna. 2006. A reference collection for web spam. SIGIR Forum 40, 2 (Dec. 2006), 11--24.

Digital Library

[14]

Charles L. A. Clarke, Nick Craswell, and Ian Soboroff. {n.d.}. Overview of the TREC 2004 terabyte track. Retrieved from https://trec.nist.gov/pubs/trec13/papers/TERA.OVERVIEW.ps.

[15]

Charles L. A, Clarke, Falk Scholer, and Ian Soboroff. {n.d.}. The TREC 2005 terabyte track.

[16]

Charles L. A. Clarke, Nick Craswell, and Ian Soboroff. 2009. Overview of the TREC 2009 web track. In Proceedings of the 18th Text REtrieval Conference (TREC’09). http://trec.nist.gov/pubs/trec18/papers/WEB09.OVERVIEW.pdf.

[17]

Charles L. A. Clarke, Nick Craswell, Ian Soboroff, and Ellen M. Voorhees. 2011. Overview of the TREC 2011 web track. In Proceedings of the 20th Text REtrieval Conference (TREC’11). http://trec.nist.gov/pubs/trec20/papers/WEB.OVERVIEW.pdf.

[18]

Charles L. A. Clarke, Nick Craswell, and Ellen M. Voorhees. 2012. Overview of the TREC 2012 web track. In Proceedings of the 21st Text REtrieval Conference (TREC’12). http://trec.nist.gov/pubs/trec21/papers/WEB12.overview.pdf.

[19]

Charles L. A. Clarke, Ian Soboroff Nick Craswell, and Gordon V. Cormack. 2010. Overview of the TREC 2010 web track. In Proceedings of the 18th Text REtrieval Conference (TREC’10). http://trec.nist.gov/pubs/trec19/papers/WEB.OVERVIEW.pdf.

[20]

Stéphane Clinchant and Eric Gaussier. 2013. A theoretical analysis of pseudo-relevance feedback models. In Proceedings of the 2013 Conference on the Theory of Information Retrieval (ICTIR’13). ACM, New York, NY.

Digital Library

[21]

Gordon V. Cormack, Mark D. Smucker, and Charles L. A. Clarke. 2011. Efficient and effective spam filtering and re-ranking for large web datasets. Inf. Retr. 14, 5 (2011), 441--465.

Digital Library

[22]

Nick Craswell and David Hawking. 2004. Overview of the TREC 2004 web track. In Proceedings of the 13th Text REtrieval Conference (TREC’04). http://trec.nist.gov/pubs/trec13/papers/WEB.OVERVIEW.pdf.

[23]

Nick Craswell, David Hawking, Ross Wilkinson, and Mingfang Wu. 2003. Overview of the TREC 2003 web track. In Proceedings of the 12th Text REtrieval Conference (TREC’03). 78--92.

[24]

Ronan Cummins. 2017. Improved query-topic models using pseudo-relevant Pólya document models. In Proceedings of the 3rd ACM International Conference on the Theory of Information Retrieval (ICTIR’17). http://dcs.gla.ac.uk/ronanc/papers/cumminsICTIR17.pdf. To appear.

Digital Library

[25]

Ronan Cummins, Jiaul H. Paik, and Yuanhua Lv. 2015. A Pólya Urn document language model for improved information retrieval. ACM Trans. Inf. Syst. 33, 4 (2015), 21:1--21:34.

Digital Library

[26]

Zhuyun Dai, Chenyan Xiong, and Jamie Callan. 2016. Query-biased partitioning for selective search. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management (CIKM’16). ACM, New York, NY, 1119--1128.

Digital Library

[27]

Mostafa Dehghani, Samira Abnar, and Jaap Kamps. 2016. The healing power of Poison: Helpful non-relevant documents in feedback. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management (CIKM’16). ACM, New York, NY, 2065--2068.

Digital Library

[28]

Mostafa Dehghani, Hamed Zamani, Aliaksei Severyn, Jaap Kamps, and W. Bruce Croft. 2017. Neural ranking models with weak supervision. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’17). ACM, New York, NY, 65--74.

Digital Library

[29]

Emanuele Di Buccio, Giorgio Maria Di Nunzio, Nicola Ferro, DK Harman, Maria Maistro, and Gianmaria Silvello. 2015. Unfolding off-the-shelf IR systems for reproducibility. In Proceedings of the SIGIR Workshop on Reproducibility, Inexplicability, and Generalizability of Results (RIGOR’15).

[30]

Faezeh Ensan and Ebrahim Bagheri. 2017. Document retrieval model through semantic linking. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (WSDM’17). ACM, New York, NY, 181--190.

Digital Library

[31]

Nicola Ferro. 2017. Reproducibility challenges in information retrieval evaluation. J. Data Inf. Qual. 8, 2 (2017), 8:1--8:4.

Digital Library

[32]

N. Ferro, F. Crestani, M. F. Moens, J. Mothe, F. Silvestri, G. M. Di Nunzio, C. Hauff, and G. Silvello (Eds.). 2016. In Proceedings of the 38th European Conference on IR Research: Advances in Information Retrieval (ECIR’16). LNCS, Vol. 9626. Springer.

[33]

Nicola Ferro, Norbert Fuhr, Kalervo Järvelin, Noriko Kando, Matthias Lippold, and Justin Zobel. 2016. Increasing reproducibility in IR: Findings from the Dagstuhl seminar on reproducibility of data-oriented experiments in e-science. ACM SIGIR Forum 50, 1 (2016), 68--82.

Digital Library

[34]

Nicola Ferro and Gianmaria Silvello. 2016. A general linear mixed models approach to study system component effects. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’16). ACM, New York, NY, 25--34.

Digital Library

[35]

Krishnendu Ghosh, Plaban Kumar Bhowmick, and Pawan Goyal. 2017. Using re-ranking to boost deep learning based community question retrieval. In Proceedings of the International Conference on Web Intelligence (WI’17). ACM, New York, NY, 807--814.

Digital Library

[36]

Mona Golestan Far, Scott Sanner, Mohamed Reda Bouadjenek, Gabriela Ferraro, and David Hawking. 2015. On term selection techniques for patent prior art search. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). ACM, New York, NY, 803--806.

Digital Library

[37]

Jiafeng Guo, Yixing Fan, Qingyao Ai, and W. Bruce Croft. 2016. A deep relevance matching model for ad-hoc retrieval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management (CIKM’16). ACM, New York, NY, 55--64.

Digital Library

[38]

Suhit Gupta, Gail Kaiser, David Neistadt, and Peter Grimm. 2003. DOM-based content extraction of HTML documents. In Proceedings of the 12th International Conference on World Wide Web (WWW’03). ACM, New York, NY, 207--214.

Digital Library

[39]

Zoltan Gyongyi and Hector Garcia-Molina. 2005. Web spam taxonomy. In Proceedings of the 1st International Workshop on Adversarial Information Retrieval on the Web (AIRWeb’05).

[40]

A. Hanbury, G. Kazai, A. Rauber, and N. Fuhr (Eds.). 2015. Proceedings of the 37th European Conference on IR Research: Advances in Information Retrieval (ECIR’15). LNCS, Vol. 9022. Springer.

[41]

Donna Harman and Chris Buckley. 2009. Overview of the reliable information access workshop. Inf. Retriev. 12, 6 (Jul. 2009), 615--641.

Digital Library

[42]

David Hawking. 2000. Overview of the TREC-9 web track. In Proceedings of the 9th Text REtrieval Conference (TREC’00).

[43]

David Hawking and Nick Craswell. 2001. Overview of the TREC-10 web track. In Proceedings of the 10th Text REtrieval Conference (TREC’01).

[44]

Hussein Hazimeh and ChengXiang Zhai. 2015. Axiomatic analysis of smoothing methods in language models for pseudo-relevance feedback. In Proceedings of the 2015 International Conference on the Theory of Information Retrieval (ICTIR’15). 141--150.

Digital Library

[45]

Ben He and Iadh Ounis. 2004. A query-based pre-retrieval model selection approach to information retrieval. In Proceedings of the International Conference on Coupling Approaches, Coupling Media and Coupling Languages for Information Retrieval (RIAO’04). 706--719. http://dl.acm.org/citation.cfm?id=2816272.2816336

Digital Library

[46]

Nasreen Abdul Jaleel, James Allan, W. Bruce Croft, Fernando Diaz, Leah S. Larkey, Xiaoyan Li, Mark D. Smucker, and Courtney Wade. 2004. UMass at TREC 2004: Novelty and HARD. In Proceedings of the Text REtrieval Conference (TREC’04).

[47]

Wessel Kraaij and Thijs Westerveld. 2000. TNO-UT at TREC-9: How different are web documents?. In Proceedings of the 9th Text REtrieval Conference (TREC’00).

[48]

Victor Lavrenko and W. Bruce Croft. 2001. Relevance based language models. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’01). ACM, New York, NY, 120--127.

Digital Library

[49]

Jimmy J. Lin, Matt Crane, Andrew Trotman, Jamie Callan, Ishan Chattopadhyaya, John Foley, Grant Ingersoll, Craig MacDonald, and Sebastiano Vigna. 2016. Toward reproducible baselines: The open-source IR reproducibility challenge. In Proceedings of the 38th European Conference on IR Research,: Advances in Information Retrieval (ECIR’16)). 408--420.

[50]

David E. Losada and Leif Azzopardi. 2008. An analysis on document length retrieval trends in language modeling smoothing. Inf. Retriev. 11, 2 (2008), 109--138.

Digital Library

[51]

Yuanhua Lv and Cheng Xiang Zhai. 2009. A comparative study of methods for estimating query language models with pseudo feedback. In Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM’09). ACM, New York, NY, 1895--1898.

Digital Library

[52]

Saptaditya Maiti, Deba P. Mandal, and Pabitra Mitra. 2011. Tackling content spamming with a term weighting scheme. In Proceedings of the 2011 Joint Workshop on Multilingual OCR and Analytics for Noisy Unstructured Text Data. ACM, New York, NY, Article 6, 5 pages.

Digital Library

[53]

Alexandros Ntoulas, Marc Najork, Mark Manasse, and Dennis Fetterly. 2006. Detecting spam web pages through content analysis. In Proceedings of the 15th International Conference on World Wide Web (WWW’06). ACM, New York, NY, 83--92.

Digital Library

[54]

Jiaul H. Paik. 2015. A probabilistic model for information retrieval based on maximum value distribution. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). ACM, New York, NY, 585--594.

Digital Library

[55]

Dae Hoon Park, Hyun Duk Kim, ChengXiang Zhai, and Lifan Guo. 2015. Retrieval of relevant opinion sentences for new products. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). ACM, New York, NY, 393--402.

Digital Library

[56]

A. F. R. Rahman, H. Alam, and R. Hartono. 2001. Content extraction from html documents. In Proceedings of the 1st International Workshop on Web Document Analysis (WDA’01). 1--4.

[57]

Fiana Raiber. 2012. Adversarial content manipulation effects. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’12). ACM, New York, NY, 993--993.

Digital Library

[58]

Fiana Raiber and Oren Kurland. 2017. Kullback-leibler divergence revisited. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR’17). ACM, New York, NY, 117--124.

Digital Library

[59]

Fiana Raiber, Oren Kurland, and Moshe Tennenholtz. 2012. Content-based relevance estimation on the web using inter-document similarities. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management (CIKM’12). ACM, New York, NY, 1769--1773.

Digital Library

[60]

Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr. 3, 4 (Apr. 2009), 333--389.

Digital Library

[61]

S. E. Robertson and S. Walker. 1994. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’94). Springer-Verlag, New York, NY, 232--241. http://dl.acm.org/citation.cfm?id=188490.188561.

Digital Library

[62]

Gerard Salton and Chris Buckley. {n.d.}. SMART Stopword list. Retrieved from http://www.lextek.com/manuals/onix/stopwords2.html.

[63]

Amit Singhal, Chris Buckley, and Mandar Mitra. 1996. Pivoted document length normalization. In Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 21--29.

Digital Library

[64]

Amit Singhal, Gerard Salton, Mandar Mitra, and Chris Buckley. 1996. Document length normalization. Inf. Process. Manage. 32, 5 (1996), 619--633.

Digital Library

[65]

Mark D. Smucker and James Allan. 2005. An Investigation of Dirichlet Prior Smoothing’s Performance Advantage. Technical Report. CIIR, U. Mass., Amherst.

[66]

Ian Soboroff. 2013. Information retrieval evaluation demo. Retrieved from https://github.com/isoboroff/trec-demo.

[67]

Nikita Spirin and Jiawei Han. 2012. Survey on web spam detection: Principles and algorithms. SIGKDD Explor. Newsl. 13, 2 (May 2012), 50--64.

Digital Library

[68]

Fei Sun, Dandan Song, and Lejian Liao. 2011. DOM based content extraction via text density. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’11). ACM, New York, NY, 245--254.

Digital Library

[69]

Tim Weninger, William H. Hsu, and Jiawei Han. 2010. CETR: Content extraction via tag ratios. In Proceedings of the 19th International Conference on World Wide Web (WWW’10). ACM, New York, NY, 971--980.

Digital Library

[70]

Craig Willis. 2017. Evaluation Framework, National Data Service—Confluence. Retrieve from https://opensource.ncsa.illinois.edu/confluence/display/NDS/Evaluation+Framework.

[71]

Chenyan Xiong and Jamie Callan. 2015. Query expansion with freebase. In Proceedings of the 2015 International Conference on the Theory of Information Retrieval (ICTIR’15). ACM, New York, NY, 111--120.

Digital Library

[72]

Peilin Yang and Hui Fang. 2016. A reproducibility study of information retrieval models. In Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval (ICTIR’16). ACM, New York, NY, 77--86.

Digital Library

[73]

Hamed Zamani and W. Bruce Croft. 2016. Estimating embedding vectors for queries. In Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval (ICTIR’16). ACM, New York, NY, 123--132.

Digital Library

[74]

Hamed Zamani and W. Bruce Croft. 2017. Relevance-based word embedding. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’17). ACM, New York, NY, 505--514.

Digital Library

[75]

ChengXiang Zhai. 2008. Statistical Language Models for Information Retrieval A Critical Review. Vol. 2. Now Publishers Inc., Hanover, MA. 137--213 pages.

Digital Library

[76]

Chengxiang Zhai and John Lafferty. 2001. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of the Annual Conference on the Special Interest Group on Information Retrieval (SIGIR’01). ACM, New York, NY, 334--342.

Digital Library

[77]

Chengxiang Zhai and John Lafferty. 2004. A study of smoothing methods for language models applied to information retrieval. ACM Trans. Inf. Syst. 22, 2 (Apr. 2004), 179--214.

Digital Library

[78]

Guoqing Zheng and Jamie Callan. 2015. Learning to reweight terms with distributed representations. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). ACM, New York, NY, 575--584.

Digital Library

Cited By

Manshadi FMostafavi SZarifzadeh S(2023)A Query-Based Weighted Document Partitioning Method for Load Balancing in Search EnginesWireless Personal Communications: An International Journal10.1007/s11277-023-10176-y129:3(1489-1511)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11277-023-10176-y
Harrigian KDredze MBaeza-Yates RWeller KPortela MSeneviratne OWeber IYasseri TBon ASrinivas SIbáñez L(2022)The Problem of Semantic Shift in Longitudinal Monitoring of Social MediaProceedings of the 14th ACM Web Science Conference 202210.1145/3501247.3531566(208-218)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1145/3501247.3531566
Breuer TKeller JSchaer PAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)ir_metadata: An Extensible Metadata Schema for IR ExperimentsProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531738(3078-3089)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531738
Show More Cited By

Index Terms

To Clean or Not to Clean: Document Preprocessing and Reproducibility
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
      1. Retrieval effectiveness
      2. Test collections
    2. Search engine architectures and scalability
      1. Search engine indexing

Recommendations

Improvement of vector space information retrieval model based on supervised learning
IRAL '00: Proceedings of the fifth international workshop on on Information retrieval with Asian languages

This paper proposes and method to improve retrieval performance of the vector space model (VSM) by utilizing user-supplied information of those documents that are relevant to the query in question. In addition to the user's relevance feedback ...
Re-examining the effects of adding relevance information in a relevance feedback environment

This paper presents an investigation about how to automatically formulate effective queries using full or partial relevance information (i.e., the terms that are in relevant documents) in the context of relevance feedback (RF). The effects of adding ...
A GA-based query optimization method for web information retrieval

By a different use of relevance feedback (the order in which the relevant documents are retrieved, the terms of the relevant documents, and the terms of the irrelevant documents) in the design of fitness function, and by introducing three different ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Journal of Data and Information Quality

Journal of Data and Information Quality Volume 10, Issue 4

Reproducibility in Information Retrieval:Tools and Infrastructures

December 2018

106 pages

ISSN:1936-1955

EISSN:1936-1963

DOI:10.1145/3289400

Editor:
Tiziana Catarci
Sapienza University of Rome, Rome, Italy

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2018

Accepted: 01 July 2018

Revised: 01 July 2018

Received: 01 October 2017

Published in JDIQ Volume 10, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
302
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)2

Reflects downloads up to 27 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Manshadi FMostafavi SZarifzadeh S(2023)A Query-Based Weighted Document Partitioning Method for Load Balancing in Search EnginesWireless Personal Communications: An International Journal10.1007/s11277-023-10176-y129:3(1489-1511)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11277-023-10176-y
Harrigian KDredze MBaeza-Yates RWeller KPortela MSeneviratne OWeber IYasseri TBon ASrinivas SIbáñez L(2022)The Problem of Semantic Shift in Longitudinal Monitoring of Social MediaProceedings of the 14th ACM Web Science Conference 202210.1145/3501247.3531566(208-218)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1145/3501247.3531566
Breuer TKeller JSchaer PAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)ir_metadata: An Extensible Metadata Schema for IR ExperimentsProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531738(3078-3089)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531738
Breuer TPest MSchaer P(2021)Evaluating Elements of Web-Based Data Enrichment for Pseudo-relevance Feedback RetrievalExperimental IR Meets Multilinguality, Multimodality, and Interaction10.1007/978-3-030-85251-1_5(53-64)Online publication date: 14-Sep-2021
https://doi.org/10.1007/978-3-030-85251-1_5
ARSLAN A(2020)ON THE USEFULNESS OF HTML META ELEMENTS FOR WEB RETRIEVALEskişehir Technical University Journal of Science and Technology A - Applied Sciences and Engineering10.18038/estubtda.61510321:1(182-198)Online publication date: 31-Mar-2020
https://doi.org/10.18038/estubtda.615103
Kathuria AGupta ASingla R(2020)A Review of Tools and Techniques for Preprocessing of Textual DataComputational Methods and Data Engineering10.1007/978-981-15-6876-3_31(407-422)Online publication date: 20-Aug-2020
https://doi.org/10.1007/978-981-15-6876-3_31
Sebastian DNugraha K(2019)Text Normalization for Indonesian Abbreviated Word Using Crowdsourcing Method2019 International Conference on Information and Communications Technology (ICOIACT)10.1109/ICOIACT46704.2019.8938463(529-532)Online publication date: Jul-2019
https://doi.org/10.1109/ICOIACT46704.2019.8938463

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents