Bandwidth Challenge
SC 2007

Planning
 

1. Equipment

The single rack of systems was comprised of a set of low-cost 1U and 2U Supermicro servers.

1U Servers  Qty 32
   Supermicro 6015URB server
 Processor  Dual Intel E5345 2.33 GHz quad-core
 RAM  8GB
 Hard Disks  4 x 750GB Western Digital SATA-II
 NIC  1 x 10GbE
 HBA
 
 1 x QLogic 2462
 (in 8 sevrers only)

 2U Servers  Qty 4
   Supermicro X7DBE+ motherboard
 Processor  Intel E5345 2.33 GHz quad-core
 RAM  8GB
 Hard Disks  12 x 750GB Western Digital SATA
 NIC  1 x 10GbE

 Cisco 6509-E  Qty 1
 Supervisor  SUP 720 with PFC3B
 LR Xenpak  5
 LR X2  3
 SR Xenpak  15
 SR X2  21
 Cisco 3560-E  Qty1

 Storage System  Data Direct Networks S2A9550,
 8 port FiberChannel storage system,
 using 80 SATA-II disks in 5 trays
 HBA  8 x QLogic 2462, Dual port 4Gb Fiber Channel

 Miscellaneous  
   
 Rack  1x 42U, 1 x 44U
 FO PatchPanel  1
 KVM  1 x 16 port

 

2. Server Configuration

Each system ran the same custom Linux 2.6.20 kernel, with patches for FAST TCP and other TCP performance tunings built in.

Based on the different hardware combination, systems were grouped into four main categories:

A) 1U Servers with Fiber Channel Storage System + Intel Oplin 10 GbE NIC

8 of  the 1U systems were outfitted with Intel Oplin single port 10GbE NIC and a QLogic  fiberchannel HBAs connected to Data Direct Networks storage system. Maximum read/write speeds between 350-400MB/s.

B) 1U Servers with Local Storage + Intel Oplin 10 GbE NIC

4 of the 1U systems were outfitted with Intel Oplin single port 10GbE NIC and local storage with RAID-0 was used to source/sink data.
Maximum read/write speeds at 200MB/s.

C) 1U Servers with Local Storage + Myricom 10 GbE NIC

20 of 1the U systems were configured with Myricom 10GbE NIC while the 4 hard drives were placed in RAID-0 configuration. Maximum read/write speeds at 200MB/s.

D) 2U Servers with Local Storage + Myricom 10 GbE NIC

The 4 x 2U servers were whitebox systems with Myricom 10GbE NIC and each system having 12 hard disks connected through Areca 1231ML RAID Controller and placed in RAID-0. Maximum read/write speeds at 500MB/s.