Innovatives Supercomputing in Deutschland
inSiDE • Vol. 11 No. 2 • Autumn 2013
current edition
archive
centers
events
download
about inSiDE
index  index prev  prev next  next

End of the HPC-FF Era

The operation of the High Performance Computer for Fusion (HPC-FF), by the Jülich Supercomputing Centre (JSC), was completed in June 2013. This computer was essentially a 1/3rd partition of the JUROPA computer, consisting of Intel Xeon X5570 nodes (Nehalem-EP). Each node comprised two processors, providing a total of 8 CPU cores attached to 24 GB of main memory. Running at 2.93 GHz clock speed the 1080 nodes of the HPC-FF partition yielded a total peak performance of 101 Teraflops. The nodes were connected via Infiniband/QDR with fat-tree technology. Lustre Version 1.8 has been chosen as a parallel filesystem for both scratch and home data. The hardware was funded in-part by Forschungszentrum Jülich and in-part through the European Fusion Development Agreement (EFDA) [1], with the member countries contributing through a joint fund. Operation of HPC-FF commenced in August 2009. Allied to the physical hardware has been a High Level Support Team (HLST) [2], which has brought an important common focus to code optimisation in a multi-core environment.

Figure 1: The HPC-FF cluster at the Jülich Supercomputing Centre.

HPC-FF satisfied the need of EFDA scientists to perform large scale simulations in connection with magnetic fusion energy issues. To reach the goal of an energy producing fusion power plant (i) the plasma energy must be contained for long enough, (ii) the plasma must be macroscopically stable, (iii) the plasma facing structures must be able to handle the high heat loads coming from the plasma, (iv) in general materials are needed which survive the loads from the high energy fusion born neutrons, and (v) the components such as the blanket modules around the plasma, in which the neutron energy is extracted as heat (to generate the electricity) and the tritium to fuel the fusion reaction is bred, need to be designed. Other components such as radio frequency antennae, used to heat the plasma, also need to be designed. Studies by EFDA scientists using HPC-FF addressed aspects of all these issues.

In the tokamak, the most developed form of magnetic fusion device, the energy is usually primarily transported by short scale length turbulent instabilities (as opposed to classical collisional processes). So the understanding of turbulence, and of the means to suppress and control it, is very important in developing optimal fusion devices. Simulation of turbulence in the tokamak, and other magnetic confinement schemes (such as the stellarator), represented the largest single use of HPC-FF. Already in fluids, turbulence has been identified as one of the most challenging problems in classical physics by Nobel prize winner Richard Feynman. However, in magnetically confined fusion plasmas, the complexity is again dramatically increased as the interaction with electromagnetic fields has to be taken into account and the low collisionality means a simplified fluid description is often not applicable. A significant effort has been devoted in general, and on HPC-FF in particular by the HLST, to optimising turbulence codes. The turbulence simulations on HPC-FF have tackled a very broad range of issues in both tokamaks and stellarators. Separately simulations have looked at turbulence on the scale lengths at which the plasma ions gyrate around the magnetic field lines and on the much smaller scale length of electron gyration, and simulations encompassing both scale lengths have also been made. Issues addressed include how flow can suppress the turbulence and how the flows themselves are generated, and in cases detailed comparisons have been made between the simulations and measurements of ion scale turbulence. Not only does turbulence affect energy containment, but it also affects particle containment and simulations of the inward pinching effect on both fuel ions and impurities (which dilute the fusion reactions) have been performed. Simulations have also looked at how turbulence affects the high energy ions, resulting from the injection of neutral particle beams to heat the plasma.

Figure 2: The colour contours (of parallel magnetic vector potential)show the linear mode structure of a microtearing instability which represents one of the most challenging turbulence types to simulate, due to its inherent multiscale character with radial structure sizes ranging from the ion-gyroradius-scale down towards the much smaller electron-gyroradius-scales. (Image courtesy of H. Doerk)

Macroscopic instabilities have also been studied on HPC-FF – such instabilities may cause a complete loss of the plasma or further enhance energy/particle losses from the plasma. The largest effort on HPC-FF was on studying edge instabilities and their control. Improved confinement in the tokamak is linked with periodic instabilities known as ELMs (Edge Localised Modes) that in future tokamaks, such as ITER, are expected to impose large transient heat loads on components surrounding the plasma. HPC-FF made possible detailed ELM simulations and also allowed the study of how they may be ameliorated. Another significant area of study was on instabilities driven by high energy ions (which are generated by sources that heat the plasma or by fusion reactions).

Figure 2: The colour contours (of parallel magnetic vector potential)show the linear mode structure of a microtearing instability which represents one of the most challenging turbulence types to simulate, due to its inherent multiscale character with radial structure sizes ranging from the ion-gyroradius-scale down towards the much smaller electron-gyroradius-scales. (Image courtesy of H. Doerk)

Density functional and molecular dynamics simulations, used to assess the effects of high energy neutrons on material properties, have also been a major user on HPC-FF. State-of-the-art ab initio electronic structure calculations have been performed by several groups to understand properties of neutron induced defects in fusion reactor materials. One of the major achievements is the study of dislocation structures in iron and tungsten, as well as the effect of alloying elements. Experimentally damage is often studied using high energy ions, as opposed to neutrons, and the use of tunnelling electron microscopes allows in-situ observations of the damage. However, it is important to understand the degree to which such experiments reproduce the real situation of neutrons impacting bulk structures, and modelling on HPC-FF has played an important role here.

Figure 2: The colour contours (of parallel magnetic vector potential)show the linear mode structure of a microtearing instability which represents one of the most challenging turbulence types to simulate, due to its inherent multiscale character with radial structure sizes ranging from the ion-gyroradius-scale down towards the much smaller electron-gyroradius-scales. (Image courtesy of H. Doerk)

Neutronics simulations (usually using Monte Carlo methods) are particularly well suited to parallelisation and important results have been obtained on HPC-FF relating to neutron shielding in ITER component designs and the associated activation of components.

Significant studies on radio frequency heating of the plasma were conducted on HPC-FF – schemes in which RF waves at frequencies that resonantly interact with ions gyrating around the magnetic field lines have been studied (this will be a primary heating means in ITER). Issues related to both the RF antenna design and how the waves couple to the plasma have been studied. Other technology related studies have focussed on blanket design and for example flows liquid metals within them.

HPC-FF has been the first shared HPC facility for the European fusion community and has allowed important progress to be made on many fronts – of which a small number of examples are presented here. Support on code development and optimisation by the High Level Support Team has also been a crucial element. Building on the success of HPC-FF, European fusion scientists now have access to a Petaflop class computer in Japan at the International Fusion Research Centre [3], under the Broader Approach [4].

References

[1] http://www.efda.org/

[2] http://www.efda-hlst.eu/

[3] http://www.iferc.org/

[4] http://fusionforenergy.europa.eu/understandingfusion/broaderapproach.aspx

•Tim Hender*
Culham Center for Fusion Energy (CCFE), Culham, UK

•Klaus Wolkersdorfer**
Jülich Supercomputing Centre (JSC)

* Tim Hender is the Fusion Programme Manager at the Culham Centre for Fusion Energy (UK) and chair of the Board that oversaw operation and CPU time allocation on HPC-FF

** Klaus Wolkersdorfer is Head of HPC systems at the Jülich Supercomputing Centre and was a permanent expert on the HPC-FF Board


top  top