Innovatives Supercomputing in Deutschland
inSiDE • Vol. 9 No. 2 • Autumn 2011
current edition
about inSiDE
index  index prev  prev next  next

JUDGE - a new GPU Cluster in Jülich

In February 2011, the IBM iDataPlex GPU cluster JUDGE (JUelich Dedicated Gpu Experiment cluster) started production. In the f irst step JUDGE consists of 54 compute nodes, 2 login and service nodes, and 2 GPFS gateway nodes. Each compute node is equipped with 2 Intel Xeon X5650 (Westmere) 6 core processors running at 2.66 Ghz, 96 GB main memory, and additionally 2 NVIDIA Tesla M2050 (Fermi) GPUs with 3 GB memory. All nodes are connected with QDR Infiniband. JUDGE uses the global parallel file system GPFS from the storage cluster JUST for home file systems and a fast work file system. The nodes are connected to JUST via 2 GPFS gateway nodes that route the Infiniband to 10G Ethernet.

JUDGE is used for simulation projects in Jülich and collaborations of the Jülich Supercomputing Centre (JSC) with internal and external partners.

In the second step JUDGE was enlarged in October by 68 additional compute nodes. The new node type is the same as in configuration the first step except for the GPU type. Each of the new nodes contains 2 NVIDIA Tesla M2070 GPUs, that are equipped with 6 GB of memory each. The new enlargement is dedicated to JuBrain, a common platform for simulations on neuroscience of the Institut für Neurowissenschaften und Medizin (INM) and JSC.

The whole system consists of 122 nodes with 244 GPUs and 11.7 TB main memory providing a peak performance of 1.16 TF/s per node and about 140 TF/s in total. With the high number of GPUs and large amount of memory, JUDGE is well-suited for memory intensive applications and programs using GPU power for acceleration and allows for scaling tests of large multi-GPU applications.

Furthermore, the system is also used for educational purposes. Several GPU programming tutorial courses have been provided to support users in porting their applications to this accelerated cluster platform. In September, a tuning workshop has been organized for advanced users in collaboration with NVIDIA. Here applications from several fields, e.g. Micromagnetism simulation, PET image reconstruction, Protein Folding and Molecular Dynamics have been optimized for JUDGE.

Figure 1: IBM iDataPlex cluster “JUDGE”

Figure 2: Back side with rear door heat exchanger

Figure 3: Compute nodes with GPU trays on front side

• Otto Büchner
Jülich Supercomputing Centre

top  top