Abstract
With the growing needs of data intensive science, such as high energy physics, and the need to share data between multiple remote computer and data centers worldwide, the necessity for high network performance to replicate large volumes (TBytes) of data between remote sites in Europe, Japan and the U.S. is imperative. Currently, most production bulk-data replication on the network utilizes multiple parallel standard (Reno based) TCP streams. Optimizing the window sizes and number of parallel stream is time consuming, complex, and varies (in some cases hour by hour) depending on network configurations and loads. We therefore evaluated new advanced TCP stacks that do not require multiple parallel streams while giving good performances on high speed long-distance network paths. In this paper, we report measurements made on real production networks with various TCP implementations on paths with different Round Trip Times (RTT) using both optimal and sub-optimal window sizes.
We compared the New Reno TCP with the following stacks: HS-TCP, Fast TCP, S-TCP, HSTCP-LP, H-TCP and Bic-TCP. The analysis will compare and report on the stacks in terms of achievable throughput, impact on RTT, intra- and inter-protocol fairness, stability, as well as the impact of reverse traffic.
We also report on some tentative results from tests made on unloaded 10 Gbps paths during SuperComputing 2003.
Similar content being viewed by others
References
L.S. Brakmo, S.W. O'Malley and L.L. Peterson, “TCP Vegas: New Techniques for Congestion Detection and Avoidance”,SIGCOMM 1994.
D. Chiu and R. Jain, “Analysis of the Increase and Decrease Algorithms for Congestion Avoidance in Computer Networks”, in Computer Networks and ISDN Systems, June 1989, pp. 1–14.
T. Dunigan, http://www.csm.ornl.gov/~dunigan/net100/
T. Dunigan, http://www.web100.org
G. Farrache. Available online: http://doc.in2p3.fr/bbftp/
W. Feng, M. Fisk, M. Gardner and E. Weigle, “Dynamic Right-Sizing: An Automated, Lightweight, and Scalable Technique for Enhancing Grid Performance”, in 7th IFIP/IEEE International Workshop, PjHSN 2002, Berlin, April 2002.
W. Feng, J. Hurwitz, H. Newman, S. Ravot, R.L. Cottrell, O.Martin, F. Coccetti, C. Jin, X.Wei and S. Low, “Optimizing 10-gigabit Ethernet for Networks of Workstations, Clusters and Grids: A Case Study”, in Supercomputing Conference 2003, Phoenix, November 2003.
S. Floyd, “Limited Slow-Start for TCP with Large Congestion Windows”, IETF Internet Draft, August 2002. draft-floyd-tcpslowstart-01.txt
S. Floyd, “Highspeed TCP for Large Congestion Windows”, IETF Internet Draft, February 2003. draft-floyd-highspeed-02.txt
“Globus Alliance”. Available online: http://www.globus.org/ datagrid/gridftp.html
L.A. Grieco and S. Mascolo, “Performance evaluation of Westwood+ TCP over WLANs with Local Error Control”,28th Annual IEEE Conference on Local Computer Networks (LCN 2003).
Y. Gu, X. Hong, M. Mazzuci and R.L. Grossman, “SABUL: A High Performance Data Transport Protocol”,IEEE Communications Letters, 2002.
T. Hacker, B. Noble and B. Athey, “Improving Throughput and Maintaining Fairness using Parallel TCP”, Submitted to IEEE INFOCOM 2004, Hong Kong, 2004.
A. Hanushevsky, A. Trunov and R.L. Cottrell, “Peer-to-peer Computing for Secure High Performance Data Copying”, in Computing in High Energy Physics, Beijing, 2001.
E. He and J. Leigh, “Reliable Blast UDP”. Available http:// www.evl.uic.edu/eric/atp/RBUDP.doc
T.H. Henderson, E. Sahouria, S.McCanne and R.H. Katz, “On Improving the Fairness of TCP Congestion Avoidance”,IEEE Globecomm Conference, 1998.
R. Hughes-Jones, P. Clarke, S. Dallison and G. Fairey, “Performance of Gigabit and 10 Gigabit Ethernet NICs with Server Quality Motherboards”, submitted for publication in High-Speed Networks and Services for Data-Intensive Grids, Special issue of Future Generation Computer Systems (FGCS), 2003.
“Indiana University Advanced Network Management Lab Tsunami Project”. Available at http://www.indiana.edu/ ~anml/anmlresearch.html
C. Jin, D.Wei, S.H. Low, G. Bushmaster, J. Bunn, D.H. Choe, R.L.A. Cottrell, J.C. Doyle, W. Feng, O. Martin, H. Newman, F. Paganini, S. Ravot and S. Singh, “FAST TCP: From Theory to Experiments”, in First International Workshop on Protocols for Fast Long-Distance Networks (PFLDNet 2003), Geneva, February 2003.
C. Jin, D.X. Wei and S.H. Low, “FAST TCP: Motivation, Architecture, Algorithms, Performance”, in IEEE INFOCOM 2004, Hong Kong, March 2004.
D. Katabi, M. Handley and C. Rohrs, “Internet Congestion Control for High Bandwidth Delay Product Network”, in ACM SIGCOMM, Pittsburgh, August 2002.
T. Kelly, “Scalable TCP: Improving Performance in HighspeedWide Area Networks”,ACMSIGCOMM, Vol. 33, No. 2, 2003, pp. 83–91.
A. Kuzmanovic and E.W. Knightly, “TCP-LP: A Distributed Algorithm for Low Priority Data Transfer”, in IEEE INFOCOM, San Francisco, April 2003.
Y. Li, URL: http://www.hep.ucl.ac.uk/~ytl/tcpip/hstcp/
Y. Li, URL: http://www.hep.ucl.ac.uk/~ytl/tcpip/linux/ txqueuelen/
Mathis, Semke, Mahdavi and Ott, “The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm”,Computer Communication Review, Vol. 27, No. 3, July 1997.
“Research and Technological Development for a Transatlantic Grid”, http://datatag.web.cern.ch/datatag/
L. Rizzo, “Dummynet: A Simple Approach to the Evaluation of Network Protocols”,ACM Computer Communications Review, Vol. 27, No. 1, pp. 31–41, 1997.
SC2003, http://www.sc-conference.org/sc2003/
R. Shorten, D. Leith, J. Foy and R. Kildu, Analysis and Design of Congestion Control in Synchronised Communication Networks, 2003.
“The Network Simulator-ns-2”. Available at http://www. isi.edu/nsnam/ns/
M. Vojnovic, J.-Y. Le Boudec and C. Boutremans, “Global Fairness of Additive-Increase and Multiplicative-Decrease with Heterogeneous Round-Trip Times”, in Proceedings of IEEE INFOCOM'2000, Tel Aviv, Israel, March 2000, pp. 1303–1312.
L. Xu, K. Harfoush and I. Rhee, “Binary Increase Congestion Control (BIC) for Fast, Long-Distance Networks”, to appear in Infocom 2004, Hong Kong, March 2004.
L. Zhang, S. Shenker and D.D. Clark, “Observations and Dynamics of a Congestion Control Algorithm: the Effects of Two-Way Traffic”, in Proceedings of ACM SIGCOMM'91, 1991. 17, pp. 133–147.
Available at http://dast.nlanr.net/Projects/lperf/
Available at http://www.hep.uvl.ac.uk/~ytl/tcpip/linux/ altaimd/
http://www.-iepm.slac.stanford.edu/bw/tpc-eval/
Rights and permissions
About this article
Cite this article
Bullot, H., Les Cottrell, R. & Hughes-Jones, R. Evaluation of Advanced TCP Stacks on Fast Long-Distance Production Networks. Journal of Grid Computing 1, 345–359 (2003). https://doi.org/10.1023/B:GRID.0000037555.53402.4f
Issue Date:
DOI: https://doi.org/10.1023/B:GRID.0000037555.53402.4f