The Distributed ASCI Supercomputer 3 (DAS-3)

More Web Proxy on the site http://driver.im/

DAS-3 Home
Achievements
Announcements
Overview
Cluster Sites
Connectivity
Research
User Accounts
Usage Policy
Job Execution
History
Steering Group
About the name...

DAS-3 Connectivity

An important innovative aspect of the DAS-3 system is the wide-area communications infrastructure between the clusters. Each cluster has 1Gbps of connectivity to the regular Internet through its local University and then on to SURFnet. To satisfy higher demands in bandwidth, transparency and latency, however, the StarPlane project collaborates with SURFnet to enable DAS-3 to use dedicated 10Gbps lightpaths across SURFnet's DWDM backbone between clusters.

The following picture shows the network connections between the clusters via the SURFnet6 Common Photonic Layer (CPL) network infrastructure, which is one of the world's first production examples of a hybrid network.

DAS-3 Local Network Architecture

The connectivity of each local cluster is shown in the following figure. There is a LAN connection to the local university and a WAN connection to SURFnet.

The head node and the cluster nodes connect to the LAN with Ethernet interfaces: 1 GE for the cluster nodes and 10GE for the head node. The Ethernet switches used for this connection are stackable Nortel 5530s and 5510s. We are particularly interested in the WAN connectivity to the SURFnet6 CPL network, as it allows the creation of lightpaths through the optical infrastructure. The WAN connections, as shown in the drawing, go via a Myrinet switch. Both the head node and the cluster nodes use 10 Gbit/s connections on NIC set to operate in Myrinet mode. The Myrinet switch itself has 8 10GE connections to SURFnet, and more specifically to a Nortel OME 6500.

Using the WAN connections to CPL the DAS-3 cluster nodes will participate in StarPlane. Applications running on the cluster will contact the management plane to request a suitable network path through SURFnet6. The StarPlane control plane will configure both the network portion in the DAS-3 domain, such as the Myrinet switches, and negotiate the path with the CPL components on SURFnet6.