Abstract
Peer-to-Peer (P2P) is an attractive paradigm for building distributed network applications. A particularly intriguing class of distributed applications consists in Information Retrieval (IR) systems. The issue of Peer-to-Peer Information Retrieval (P2PIR) is being tackled by researchers attempting to provide valuable insights and to propose solutions to use it successfully. Nearly, all published studies have been evaluated by simulation means, using well-known document collections (usually acquired from TREC). This practice leads to two problems: First, there is little justification in favour of the document distributions used by relevant studies and second, since different studies use different experimental benchmarks, there is no common ground for comparing the solutions proposed. In this paper, we propose Peer-to-Peer Information Retrieval Benchmarking (P2PIRB), a benchmarking framework for P2PIR. P2PIRB allows to distribute documents and queries according to various ways. This work marks the start of an effort to provide more realistic evaluation environments for P2PIR systems as well as to create a common ground to compare the current and future architectures.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Dekhtyar, A., Hayes, J.H.: Good Benchmarks are Hard To Find: Toward the Benchmark for Information Retrieval Applications in Software Engineering. In: ICSM 2006 Working Session: Information Retrieval Based Approaches in Software Evolution (2007)
Harman, D.K.: Overview of the first Text REtrieval Conference (TREC1). In: Proceedings of the First Text REtrieval Conference (TREC1), pp. 207–500. NIST Special Publication (1993)
Gerard, S.: The SMART Retrieval System—Experiments in Automatic Document Retrieval. Prentice Hall Inc., Englewood Cliffs (1971)
Cleverdon, C.W.: The Cranfield tests on index language devices, Aslib Proceedings, pp. 173–192. Aslib Proceedings Prentice Hall Inc., Englewood Cliffs (1967)
Aberer, K., Cudré-Mauroux, P., Hauswirth, M., Gridvine, T.V.: Pelt: Building internet-scale semantic overlay networks, In: ISWC (2004)
Idreos, S., Koubarakis, M., Tryfonopoulos, C.: P2p-diet: An extensible p2p service that unifies ad-hoc and continuous querying in super-peer networks, In: SIGMOD (2004)
Nottelmann, H., Fischer, G., Titarenko, A., Nurzenski, A.: An integrated approach for searching and browsing in heterogeneous peer-to-peer networks. In: HDIR (2005)
Bender, M., Michel, S., Triantafillou, P., Weikum, G., Zimmer, C.: Minerva: Collaborative p2p search. In: VLDB, pp. 1263–1266 (2005)
Cuenca-Acuna, F.M., Peery, C., Martin, R.P., Nguyen, T.D.: Planetp: Using gossiping to build content addressable peer-to-peer information sharing communities. In: HPDC (2003)
Suel, T., Mathur, C., wen Wu, J., Zhang, J., Delis, A., Kharrazi, M., Long, X., Shanmugasundaram, K.: Odissea: A peer-to-peer architecture for scalable web search and information retrieval. In: WebDB (2003)
Lu, J., Callan, J.: Content-based retrieval in hybrid peer-to-peer networks. In: CIKM ’03: Proceedings of the twelfth international conference on Information and knowledge management, New Orleans, LA, USA, pp. 199–206 (2003)
Klampanos, I.A., Jose, J.M., Poznanski, V., Dickman, P.: A Suite of testbeds for the Realistic Evaluation of Peer-to-Peer Information Retrieval Systems. In: 27th European Conference on IR Research, Santiago de Compostela, Spain, March 21-23, pp. 38–51 (2005)
Bawa, M., Manku, G.S., Raghavan, P.: SETS: search enhanced by topic segmentation. In: SIGIR ’03: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in information retrieval, Toronto, Canada, pp. 306–313 (2003)
Neumann, T., Bender, M., Michel, S., Weikum, G.: A Reproducible Benchmark for P2P Retrieval. In: Proc. of the First International Workshop on Performance and Evaluation of Data Management Systems. In: ExpDB 2006 at ACM SIGMOD, Chicago, pp. 1–8 (2006)
Cooper, B.F.: A content model for evaluating peer-to-peer searching techniques In: ACM/IFIP/USENIX 5th International Middleware Conference, Toronto, Ontario, Canada, pp. 18–22 (October 2004)
Crespo, A., Garcia-Molina, H.: Semantic Overlay Networks for P2P Systems, Technical report, Computer Science Department (2002)
Schlosser, M., Condie, T., Kamvar, S.: Simulating A File-Sharing P2P Network First Workshop on Semantics in P2P and Grid Computing, Budapest, Hungary (May 20, 2003)
Lv, Q., Cao, P., Cohen, E., Li, K., Shenker, S.: Search and replication in unstructured peerto- peer networks. In: ICS, NewYork, USA (2002)
Adamic, L., Huberman, B.: Zipf’s law and the Internet. In: Glottometrics vol. 3 (2002)
Druschel, P., Rowstron, A.: PAST: A large-scale, persistent peer-to-peer storage utility. In: Proceedings of HOTOS, pp. 75–80 (2001)
Jelasity, M., Montresor, A., Jesi, J.P., Voulgaris, S.: The Peersim simulator, http://peersim.sf.net
RARE project (2008), http://www-inf.int-evry.fr/defude/RARE
Gnutella (2009), ”Gnutella website”, http://www.gnutella.com/
Yeferny, T., Arour, K., Slimani, Y.: Routage semantique des requêtes dans les systèmes pair-à-pair. In: CORIA, pp. 131–147 (2009)
Soboroff, I.: Does WT10g look like the web. In: Proc. ACM-SIGIR Int. Conf. on Research and Development in Information Retrieval, pp. 423–424. ACM Press, New York (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zammali, S., Arour, K. (2010). P2PIRB: Benchmarking Framework for P2PIR . In: Hameurlain, A., Morvan, F., Tjoa, A.M. (eds) Data Management in Grid and Peer-to-Peer Systems. Globe 2010. Lecture Notes in Computer Science, vol 6265. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15108-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-15108-8_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15107-1
Online ISBN: 978-3-642-15108-8
eBook Packages: Computer ScienceComputer Science (R0)