More Web Proxy on the site http://driver.im/

Article

Jellyfish: networking data centers randomly

Authors:

P. Brighten GodfreyAuthors Info & Claims

NSDI'12: Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation

Page 17

Published: 25 April 2012 Publication History

Abstract

Industry experience indicates that the ability to incrementally expand data centers is essential. However, existing high-bandwidth network designs have rigid structure that interferes with incremental expansion. We present Jellyfish, a high-capacity network interconnect which, by adopting a random graph topology, yields itself naturally to incremental expansion. Somewhat surprisingly, Jellyfish is more cost-efficient than a fat-tree, supporting as many as 25% more servers at full capacity using the same equipment at the scale of a few thousand nodes, and this advantage improves with scale. Jellyfish also allows great flexibility in building networks with different degrees of oversubscription. However, Jellyfish's unstructured design brings new challenges in routing, physical layout, and wiring. We describe approaches to resolve these challenges, and our evaluation suggests that Jellyfish could be deployed in today's data centers.

References

[1]

CPLEX Linear Program Solver. http://www-01. ibm.com/software/integration/optimization/ cplex-optimizer/.

[2]

An implementation of k-shortest path algorithm. http://code. google.com/p/k-shortest-paths/.

[3]

Project blackbox. http://www.sun.com/emrkt/blackbox/ story.jsp.

[4]

Rackable systems. ICE Cube modular data center. http://www. rackable.com/products/icecube.aspx.

[5]

SGI ICE Cube Air expandable line of modular data centers. http://sgi.com/products/data_center/ice_cube_air.

[6]

M. Al-Fares, A. Loukissas, and A. Vahdat. A scalable, commodity data center network architecture. In SIGCOMM, 2008.

Digital Library

[7]

L. N. Bhuyan and D. P. Agrawal. Generalized hypercube and hyperbus structures for a computer network. IEEE Transactions on Computers, 1984.

Digital Library

[8]

B. Bollobás. The isoperimetric number of random regular graphs. Eur. J. Comb., 1988.

Digital Library

[9]

B. Bollobás. Random graphs, 2nd edition. 2001.

[10]

B. Bollobás and W. F. de la Vega. The diameter of random regular graphs. In Combinatorica 2, 1981.

[11]

A. Broder and E. Shamir. On the second eigenvalue of random regular graphs. In FOCS, 1987.

Digital Library

[12]

F. Comellas and C. Delorme. The (degree, diameter) problem for graphs. http://maite71.upc.es/grup_de_grafs/table_ g.html/.

[13]

A. R. Curtis, T. Carpenter, M. Elsheikh, A. Lopez-Ortiz, and S. Keshav. REWIRE: an optimization-based framework for unstructured data center network design. In INFOCOM, 2012.

[14]

A. R. Curtis, S. Keshav, and A. Lopez-Ortiz. LEGUP: using heterogeneity to reduce the cost of data center network upgrades. In ACM CoNEXT, 2010.

Digital Library

[15]

Digital Reality Trust. What is driving the us market? http: //goo.gl/qiaRY, 2001.

[16]

Facebook. Facebook to expand Prineville data center. http: //goo.gl/fJAoU.

[17]

N. Farrington, G. Porter, S. Radhakrishnan, H. H. Bazzaz, V. Subramanya, Y. Fainman, G. Papen, and A. Vahdat. Helios: A hybrid electrical/optical switch architecture for modular data centers. In SIGCOMM, 2010.

Digital Library

[18]

A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. VL2: a scalable and flexible data center network. In SIGCOMM, 2009.

Digital Library

[19]

C. Guo, G. Lu, D. Li, H.Wu, X. Zhang, Y. Shi, C. Tian, Y. Zhang, and S. Lu. BCube: A high performance, server-centric network architecture for modular data centers. In SIGCOMM, 2009.

Digital Library

[20]

C. Guo, H. Wu, K. Tan, L. Shi, Y. Zhang, and S. Lu. DCell: a scalable and fault-tolerant network structure for data centers. In SIGCOMM, 2008.

Digital Library

[21]

L. Gyarmati and T. A. Trinh. Scafida: A scale-free network inspired data center architecture. In SIGCOMM CCR, 2010.

Digital Library

[22]

J. Hamilton. Datacenter networks are in my way. http://goo. gl/Ho6mA.

[23]

HP. HP EcoPOD. http://goo.gl/8A0Ad.

[24]

HP. Pod 240a data sheet. http://goo.gl/axHPp.

[25]

R. K. Jain, D.-M. W. Chiu, and W. R. Hawe. A quantitative measure of fairness and discrimination for resource allocation in shared computer systems. Technical report, Digital Equipment Corporation, 1984.

[26]

J. Kim, W. J. Dally, S. Scott, and D. Abts. Technology-driven, highly-scalable dragonfly topology. ACM SIGARCH, 2008.

Digital Library

[27]

F. T. Leighton. Introduction to parallel algorithms and architectures: Arrays, trees, hypercubes. 1991.

Digital Library

[28]

A. Licis. Data center planning, design and optimization: A global perspective. http://goo.gl/Sfydq.

[29]

B. D. McKay and N. C. Wormald. Uniform generation of random regular graphs of moderate degree. J. Algorithms, 1990.

Digital Library

[30]

N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S. Shenker, and J. Turner. OpenFlow: enabling innovation in campus networks. SIGCOMM CCR, 2008.

Digital Library

[31]

A. B. Michael, M. Nolle, and G. Schreiber. A message passing model for communication on random regular graphs. In International Parallel Processing Symposium (IPPS), 1996.

[32]

Microsoft. Link layer topology discovery protocol. http:// goo.gl/bAcZ5.

[33]

R. Miller. Facebook now has 30,000 servers. http://goo.gl/ EGD2D.

[34]

R. Miller. Facebook server count: 60,000 or more. http:// goo.gl/79J4.

[35]

J. C. Mogul, J. Tourrilhes, P. Yalagandula, P. Sharma, A. R. Curtis, and S. Banerjee. DevoFlow: cost-effective flow management for high performance enterprise networks. In Hotnets, 2010.

Digital Library

[36]

J. Mudigonda, P. Yalagandula, M. Al-Fares, and J. C. Mogul. SPAIN: COTS data-center ethernet for multipathing over arbitrary topologies. In NSDI, 2010.

Digital Library

[37]

J. Mudigonda, P. Yalagandula, and J. Mogul. Taming the flying cable monster: A topology design and optimization framework for data-center networks. 2011.

[38]

R. N. Mysore, A. Pamboris, N. Farrington, N. Huang, P. Miri, S. Radhakrishnan, V. Subramanya, and A. Vahdat. Portland: A scalable fault-tolerant layer 2 data center network fabric. In SIGCOMM, 2009.

Digital Library

[39]

L. Popa, S. Ratnasamy, G. Iannaccone, A. Krishnamurthy, and I. Stoica. A cost comparison of datacenter network architectures. In ACM CoNEXT, 2010.

Digital Library

[40]

E. Rosen, A. Viswanathan, and R. Callon. Multiprotocol Label Switching Architecture. RFC 3031, 2001.

Digital Library

[41]

J.-Y. Shin, B. Wong, and E. G. Sirer. Small-world datacenters. ACM Symposium on Cloud Computing (SOCC), 2011.

Digital Library

[42]

A. Singla, A. Singh, K. Ramachandran, L. Xu, and Y. Zhang. Proteus: a topology malleable data center network. In HotNets, 2010.

Digital Library

[43]

G. Wang, D. G. Andersen, M. Kaminsky, K. Papagiannaki, T. S. E. Ng, M. Kozuch, and M. Ryan. c-Through: Part-time optics in data centers. In SIGCOMM, 2010.

Digital Library

[44]

D. Wischik, C. Raiciu, A. Greenhalgh, and M. Handley. Design, implementation and evaluation of congestion control for Multi-path TCP. In NSDI, 2011.

Digital Library

[45]

H. Wu, G. Lu, D. Li, C. Guo, and Y. Zhang. MDCube: a high performance network structure for modular data center interconnection. In ACM CoNEXT, 2009.

Digital Library

[46]

J. Yen. Finding the k shortest loopless paths in a network. Management Science, 1971.

Cited By

Gheissi SMahmood SGhorbani SRodriguez NCrowcroft J(2022)Burstiness in data center topologiesProceedings of the 3rd International CoNEXT Student Workshop10.1145/3565477.3569160(29-31)Online publication date: 9-Dec-2022
https://dl.acm.org/doi/10.1145/3565477.3569160
Schweiger OFoerster KSchmid S(2021)Improving the Resilience of Fast Failover RoutingProceedings of the Symposium on Architectures for Networking and Communications Systems10.1145/3493425.3502747(1-7)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.1145/3493425.3502747
Bienkowski MFuchssteiner DMarcinkowski JSchmid S(2021)Online Dynamic B-MatchingACM SIGMETRICS Performance Evaluation Review10.1145/3453953.345397648:3(99-108)Online publication date: 5-Mar-2021
https://dl.acm.org/doi/10.1145/3453953.3453976
Show More Cited By

Jellyfish: networking data centers randomly
1. Networks
  1. Network properties

Recommendations

A new routing scheme for Jellyfish and its performance with HPC workloads
SC '13: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

The jellyfish topology where switches are connected using a random graph has recently been proposed for large scale data-center networks. It has been shown to offer higher bisection bandwidth and better permutation throughput than the corresponding fat-...
The Flow Induced by a Jellyfish

The purpose of this study is to understand the propulsion mechanism of a jellyfish during its swimming. We observed the motion of a jellyfish (Aurelia aurita) by a motion-capture camera, and measured the vector field of flow around a jellyfish by using ...
Research on multiple jellyfish classification and detection based on deep learning
Abstract
In recent years, there have been frequent jellyfish outbreaks in many offshore areas worldwide, which have severely affected marine fishery production, coastal tourism, coastal industrial cooling water systems, and marine ecology. Achieving the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

NSDI'12: Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation

April 2012

30 pages

Program Chairs:
Steven Gribble
University of Washington
,
Dina Katabi
Massachusetts Institute of Technology

Sponsors

VMware
NSF: National Science Foundation
Google Inc.
Infosys
Microsoft Reasearch: Microsoft Reasearch

Publisher

USENIX Association

United States

Publication History

Published: 25 April 2012

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

115
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gheissi SMahmood SGhorbani SRodriguez NCrowcroft J(2022)Burstiness in data center topologiesProceedings of the 3rd International CoNEXT Student Workshop10.1145/3565477.3569160(29-31)Online publication date: 9-Dec-2022
https://dl.acm.org/doi/10.1145/3565477.3569160
Schweiger OFoerster KSchmid S(2021)Improving the Resilience of Fast Failover RoutingProceedings of the Symposium on Architectures for Networking and Communications Systems10.1145/3493425.3502747(1-7)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.1145/3493425.3502747
Bienkowski MFuchssteiner DMarcinkowski JSchmid S(2021)Online Dynamic B-MatchingACM SIGMETRICS Performance Evaluation Review10.1145/3453953.345397648:3(99-108)Online publication date: 5-Mar-2021
https://dl.acm.org/doi/10.1145/3453953.3453976
Khani MGhobadi MAlizadeh MZhu ZGlick MBergman KVahdat AKlenk BEbrahimi EKuipers FCaesar M(2021)SiP-MLProceedings of the 2021 ACM SIGCOMM 2021 Conference10.1145/3452296.3472900(657-675)Online publication date: 9-Aug-2021
https://dl.acm.org/doi/10.1145/3452296.3472900
Mellette WDas RGuo YMcGuinness RSnoeren APorter GBhagwan RPorter G(2020)Expanding across time to deliver bandwidth efficiency and low latencyProceedings of the 17th Usenix Conference on Networked Systems Design and Implementation10.5555/3388242.3388244(1-18)Online publication date: 25-Feb-2020
https://dl.acm.org/doi/10.5555/3388242.3388244
Nguyen CKieu CNguyen K(2020)Improved Compact Routing Schemes for Random InterconnectsInternational Journal of Distributed Systems and Technologies10.4018/IJDST.202007010511:3(89-109)Online publication date: 1-Jul-2020
https://dl.acm.org/doi/10.4018/IJDST.2020070105
Harsh VJyothi SGodfrey PZhao BZheng HMadhyastha HPadmanabhan V(2020)Spineless Data CentersProceedings of the 19th ACM Workshop on Hot Topics in Networks10.1145/3422604.3425945(67-73)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.1145/3422604.3425945
Zhang MMysore RSupittayapornpong SGovindan RLorch JYu M(2019)Understanding lifecycle management complexity of datacenter topologiesProceedings of the 16th USENIX Conference on Networked Systems Design and Implementation10.5555/3323234.3323255(235-254)Online publication date: 26-Feb-2019
https://dl.acm.org/doi/10.5555/3323234.3323255
Zhao SWang RZhou JOng JMogul JVahdat ALorch JYu M(2019)Minimal rewiringProceedings of the 16th USENIX Conference on Networked Systems Design and Implementation10.5555/3323234.3323254(221-234)Online publication date: 26-Feb-2019
https://dl.acm.org/doi/10.5555/3323234.3323254
Zilberman NBracha GSchzukin GLorch JYu M(2019)StardustProceedings of the 16th USENIX Conference on Networked Systems Design and Implementation10.5555/3323234.3323247(141-159)Online publication date: 26-Feb-2019
https://dl.acm.org/doi/10.5555/3323234.3323247
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents