Characterising a grid site's traffic
Pages 707 - 716
Abstract
Grid computing has been widely adopted for intensive high performance computing. Since grid resources are distributed over complex large-scale infrastructures, understanding grid site data traffic behaviour is important for efficient resource utilisation, performance optimisation, and the design of future grid sites as well as traffic-aware grid applications. In this paper, we study and analyse the traffic generated at a grid site in the Large Hadron Collider (LHC) Computing Grid (LCG). We find that most of the generated traffic is TCP-based and that a small set of grid applications generate significant amounts of the data. Upon analysing the different traffic metrics, we also find that the traffic exhibits long-range dependence and self-similarity. We also investigate packet-level metrics such as throughput, packet rate, round trip time (RTT) and packet loss. Our study establishes that these metrics can be well represented by Gaussian mixture models. The findings we present in this paper will enable accurate grid site traffic monitoring and potentially on-the-fly traffic modelling and prediction. It will also lead to a better understanding of grid site's traffic behaviour and contribute to more efficient grid site planning, traffic management, data transmission protocol optimisation, and data-aware grid application design.
References
[1]
}}Berkeley Database Information Index. https://twiki.cern.ch/twiki//bin/view/EGEE/BDII.
[2]
}}Globus Resource Allocation Manager (GRAM). https://www.globus.org/toolkit/docs/2.4/gram.
[3]
}}Ganglia. https://ganglia.sourceforge.net, April 2003.
[4]
}}Europe-china grid internetworking (ec-gin), fp6 strep project, contract number 045256. http://www.ec-gin.eu/, 2006.
[5]
}}W. Allcock, J. Bester, J. Bresnahan, A. Chervenak, L. Liming, S. Meder, and S. Tuecke. GridFTP Protocol Specification. GGF GridFTP Working Group Document, Sep, 2002.
[6]
}}E. Altman, K. Avrachenkov, and C. Barakat. A stochastic model of tcp/ip with stationary random losses. IEEE/ACM Trans. Netw., 13(2):356--369, 2005.
[7]
}}S. Andreozzi, S. Burke, L. Field, S. Fisher, B. Konya, M. Mambelli, J. Schopf, M. Viljoen, and A. Wilson. Glue schema specification version 1.2. 2005.
[8]
}}P. Barford and M. Crovella. Generating representative web workloads for network and server performance evaluation. ACM SIGMETRICS Performance Evaluation Review, 26(1):151--160, 1998.
[9]
}}J. Beran, R. Sherman, M. Taqqu, and W. Willinger. Long-range Dependence in Variable-bit-rate Video Traffic. IEEE Trans. on Comm., 43(234):1566--1579, 1995.
[10]
}}M. Borella. Source models of network game traffic. Computer Communications, 23(4):403--410, 2000.
[11]
}}M. Crovella and A. Bestavros. Self-similarity in World Wide Web traffic: Evidence and Possible Causes. IEEE/ACM Transactions on networking, 5(6):835--846, 1997.
[12]
}}Y. El-khatib and C. Edwards. A survey-based study of grid traffic. In Proceedings of the International Conference on Networks for Grid Applications (GridNets '07), 2007.
[13]
}}D. Ersoz, M. Yousif, and C. Das. Characterizing Network Traffic in a Cluster-based, Multi-tier Data Center. In Distributed Computing Systems, 2007. ICDCS'07. 27th International Conference on, pages 59--59, 2007.
[14]
}}W. Feng and P. Tinnakornsrisuphap. The Failure of TCP in High-performance Computational Grids. In Supercomputing, ACM/IEEE 2000 Conference, pages 37--37, 2000.
[15]
}}A. Field, U. Harder, and P. Harrison. Network Traffic Measurements in a Switched Ethernet Environment. In UKPEW2002, pages 47--58, June 2002.
[16]
}}A. Field, U. Harder, and P. Harrison. Measurement and Modelling of Self-similar Traffic in Computer Networks. IEE Proceedings-Communications, 151(4):355--363, 2004.
[17]
}}D. Figueiredo, B. Liu, A. Feldmann, V. Misra, D. Towsley, and W. Willinger. On TCP and Self-similar Traffic. Performance Evaluation, 61(2--3):129--141, 2005.
[18]
}}L. Gleser and D. Moore. The effect of dependence on chi-squared and empiric distribution tests of fit. The Annals of Statistics, 11(4):1100--1108, 1983.
[19]
}}T. Karagiannis, M. Molle, and M. Faloutsos. Long-range Dependence: Ten Years of Internet Traffic Modeling. IEEE Internet Computing, 8(5):57--64, 2004.
[20]
}}T. Karagiannis, M. Molle, M. Faloutsos, and A. Broido. A nonstationary Poisson view of Internet traffic. In IEEE INFOCOM, volume 3, pages 1558--1569. Citeseer, 2004.
[21]
}}N. Kourtellis, L. Prieto, A. Iamnitchi, G. Zarrate, and D. Fraser. Data transfers in the grid: workload analysis of globus gridftp. In DADC '08: Proceedings of the 2008 international workshop on Data-aware distributed computing, pages 29--38, New York, NY, USA, 2008. ACM.
[22]
}}A. Kumar. Comparative performance analysis of versions of TCP in a local network with a lossy link. IEEE/ACM Transactions on Networking (TON), 6(4):485--498, 1998.
[23]
}}J. Lakkakorpi, A. Heiner, and J. Ruutu. Measurement and characterization of Internet gaming traffic. In Research Seminar on Networking, Helsinki University of Technology, Networking Laboratory, Espoo, Finland. Citeseer, 2002.
[24]
}}Z. Liu, N. Niclausse, and C. Jalpa-Villanueva. Traffic model and performance evaluation of web servers. Performance Evaluation, 46(2--3):77--100, 2001.
[25]
}}J. Padhye, V. Firoiu, D. Towsley, and J. Kurose. Modeling tcp throughput: a simple model and its empirical validation. SIGCOMM Comput. Commun. Rev., 28(4):303--314, 1998.
[26]
}}R. Pang, M. Allman, M. Bennett, J. Lee, V. Paxson, and B. Tierney. A First Look at Modern Enterprise Traffic. In Proc. Internet Measurement Conference, pages 15--28, 2005.
[27]
}}V. Paxson. Empirically derived analytic models of wide-area tcp connections. IEEE/ACM Trans. Netw., 2(4):316--336, 1994.
[28]
}}V. Paxson. End-to-end routing behavior in the Internet. ACM SIGCOMM Computer Comm. Review, 36(5):56, 2006.
[29]
}}V. Paxson and S. Floyd. Wide Area Traffic: The Failure of Poisson Modeling. IEEE/ACM Transactions on Networking (TON), 3(3):226--244, 1995.
[30]
}}S. Pederson and M. Johnson. Estimating model discrepancy. Technometrics, 32(3):305--314, 1990.
[31]
}}W. Stallings. High-speed Networks: TCP/IP and ATM Design Principles. Prentice-Hall, Inc., NJ, USA, 1997.
[32]
}}W. Willinger, M. Taqqu, R. Sherman, and D. Wilson. Self-similarity Through High-variability: Statistical Analysis of Ethernet LAN Traffic at the Source Level. IEEE/ACM Transactions on Networking (TON), 5(1):71--86, 1997.
[33]
}}D. Xu. The applications of mixtures of normal distributions in empirical finance: A selected survey. Working Papers 0904, Univ. of Waterloo, Dept. of Economics, Sept. 2009.
[34]
}}E. Yildirim, D. Yin, and T. Kosar. Balancing tcp buffer vs parallel streams in application level throughput optimization. In DADC '09: Proceedings of the second international workshop on Data-aware distributed computing, pages 21--30, New York, NY, USA, 2009. ACM.
Index Terms
- Characterising a grid site's traffic
Recommendations
MGC middleware for grid computing: the Globus Toolkit
ACAI '11: Proceedings of the International Conference on Advances in Computing and Artificial IntelligenceGrid computing has made substantial advances during the last decade. A major concern in Grid environments is dealing with the high degree of heterogeneity of resources that can range from laptops and PCs to supercomputers. The unified virtual view of ...
The Grid Resource Broker workflow engine
2nd International Workshop on Workflow Management and Applications in Grid Environments (WaGe2007)Increasingly, complex scientific applications are structured in terms of workflows. These applications are usually computationally and-or data intensive and thus are well suited for execution in grid environments. Distributed, geographically spread ...
DIRAC Site Director: Improving Pilot-Job provisioning on grid resources
AbstractTo study the constituents of matter, CERN mainly relies on the Worldwide LHC Computing Grid (WLCG), which processes petabytes of data coming from the Large Hadron Collider (LHC). LHC experiments have adopted the Pilot-Job paradigm, and deliver ...
Highlights- Submitting and monitoring distributed Pilot-Jobs is a long and complex process.
- We set up multi-threading to communicate with remote computing resources.
- We better share the workload between execution cycles.
- Our approach ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2010 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
- University of Arizona: University of Arizona
- SIGARCH: ACM Special Interest Group on Computer Architecture
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 June 2010
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
HPDC '10
Sponsor:
- University of Arizona
- SIGARCH
HPDC '10: The 19th International Symposium on High Performance Distributed Computing
June 21 - 25, 2010
Illinois, Chicago
Acceptance Rates
Overall Acceptance Rate 166 of 966 submissions, 17%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 133Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 24 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in