GCS: Delivering 10 Years of Integrated HPC Excellence for Germany
In 2007, leadership from the three leading supercomputing centres—the High-Performance Computing Center Stuttgart (HLRS), the Jülich Supercomputing Centre (JSC), and the Leibniz Computing Centre of the Bavarian Academy of Sciences in Garching near Munich (LRZ)—agreed to combine their facilities and expertise, and, supported by the German Federal Ministry of Education and Research, and the science ministries of the states of Baden-Württemberg, Bavaria, and North Rhine-Westphalia, joined forces and founded the non-profit Gauss Centre for Supercomputing (GCS).
Over 10 years, the three major German supercomputing centres have been able to learn how to collaborate and integrate their vast intellectual and technological resources into a unified German supercomputing strategy.
This integration allowed the German High Performance Computing (HPC) community to deliver multiple generations of Europe’s most powerful supercomputers, classified as Tier-0 systems, and allowed GCS to offer scientists and engineers in Germany and Europe a diverse range of computing architectures and domain expertise to address some of the world’s most challenging problems related to public health, fundamental research, climate change, urban planning, new materials, scientific engineering, and green energy, among other key research areas.
Bringing three large institutions together to share expertise and coordinate research activities required a herculean effort, and was not without risk. Working together, the directors of the three centres and their managing academic institutions helped integrate their respective manpower and computational resources to complement one another, offering users across a variety of scientific disciplines more tailored resources that would best work for their needs.
Indeed, GCS was named after one of Germany’s most famous problem-solvers, Carl Friedrich Gauss, a mathematician heralded as the father of numerical problem-solving and one who used practical solutions to answer long-standing questions.
Funding and Collaboration
GCS is jointly funded by the German Federal Ministry of Education and Research, the three states housing the three supercomputing centres, and three centres’ managing institutions: The University of Stuttgart, the Bavarian Academy of Sciences, and Forschungszentrum Jülich.
The collective power of these three centres makes Germany Europe’s HPC leader, and one of the strongest global players in HPC. The GCS structure enables the three respective centres to upgrade their machines in a “round-robin” fashion. This “life-cycle management” for the centres’ machines ensures that each centre can install a new machine every 3–4 years and GCS will constantly have a machine at the forefront of hardware and software technology. This type of close collaboration between three national German HPC centres is unique in relation to national HPC strategies around the world.
Other than integrating the procurement process at the three centres, GCS also organized integrated access methods, allowing researchers to submit a single proposal. These proposals then go through a rigorous peer review process to assess their scientific merit, then researchers get matched with the centre best-suited to their needs.
The centres work together on training courses by sharing best practices, dividing the task of training users on general HPC themes, and offering specialty training on centre-specific topics. In addition, the centres agree to work together on their outreach activities to better articulate the value that supercomputers bring to society for both government stakeholders and the public. In addition to three large-scale supercomputers, the centres offer researchers a variety of smaller clusters, data storage and analytics facilities, and sophisticated visualization resources to meet researchers’ needs. Between the collaborative spirit of sharing knowledge and training activities, and the wide array of computational resources, GCS has consistently offered German and European researchers leading-edge systems, facilities, and support for the last decade.
While all three centres collaborate with researchers from across the scientific spectrum, each one does have certain specialties.
JSC is renowned for fundamental research, physics, and neuroscience, and has primarily built ultra-high-end machines with specialized architectures to support these fields. In fact, JSC’s JUGENE machine—which, in 2007, appeared at number 2 on the biannual Top500 list of the world’s fastest supercomputers—held Germany’s highest-ever position on the list.
LRZ strongly supports geoscience, life sciences, and astrophysics research, and has been a leader in energy efficiency for machines. LRZ consistently has one of the most energy efficient supercomputers in the world, and is capable of using far higher water temperatures to cool its SuperMUC machine—using water up to 40 degrees Celsius—than the 16 degrees common in other large-scale machines.
Due it being situated in one of Germany’s industrial hotbeds, HLRS naturally has a strong focus on engineering and industrial applications. While the centre regularly hosts one of the top 20 fastest machines in the world, it focuses on bridging the gap between cutting-edge computing technology and the commercial and industrial worlds. In addition to academic partners, HLRS has official research partnerships with automaker Porsche and German IT giant T-Systems, and holds the world record for the fastest commercial application, helping ANSYS scale its Fluent code to more than 172,000 compute cores.
The three centres’ diverse capabilities and specializations have led to breakthroughs across science and engineering spectrum.
During a 2016 “extreme scaling workshop,” a Pan-European research team led by University College London Professor Peter Coveney used LRZ’s SuperMUC system to make a breakthrough in personalized medicine.
Many drug companies have turned to simulation to save time and money in developing next-generation medications. Researchers can quickly simulate thousands (or millions) of combinations of molecules and human proteins, while simultaneously calculating the likelihood the two will bind together.
The Coveney team ran simulations on the SuperMUC system for 37 hours straight—using nearly all of SuperMUC’s 250,000 compute cores—combing through many combinations of proteins binding with common breast cancer drugs in the process. The team’s research helped create a roadmap for how simulation can quickly advance medicines from trials to market, anticipate a drug’s effectiveness for individual patients, or anticipate possible side-effects.
In 2015, researchers at the University of Hohenheim used HLRS’ Hornet system to run climate simulations of the Earth’s northern hemisphere at extremely precise scales.
In order for simulation to be able to accurately predict or accurately simulate past weather events, climate researchers must divide their respective maps into a very fine grid—each cube must be no larger than 20 square kilometres for researchers to be able to simulate the small-scale structures that contribute to weather events.
The Hohenheim team, led by Professor Volker Wulfmeyer, more than half of Hornet capability and accurately simulated the Soulik typhoon—a category 4 storm (meaning that wind speeds exceed 58 metres per second) that developed in the Pacific Ocean during the summer of 2013.
Researchers studying elementary particle physics—one of the most difficult science domains for experimental observation—have increasingly turned to supercomputers to make breakthrough discoveries.
In 2015, a University of Wuppertal team led by Professor Zoltán Fodor used JSC’s JUQUEEN system for a breakthrough within quantum chromodynamics (QCD). The field of QCD is focused on understanding foundational building blocks of our universe—the constituent particles of protons and neutrons, called quarks, gluons. These simulations requires precision—a proton is only .14 percent lighter than a neutron, but even the slightest difference in mass would have had major consequences for how galaxies, and in turn, our world, formed.
The Fodor team’s simulations, published in Science, helped validate the strong force—one of the four fundamental interactions in the universe, and the force that holds protons and neutrons together to form atomic nuclei. Shortly after publishing its work, the team was lauded by Massachusetts Institute of Technology Professor Frank Wilczek—a 2004 Nobel Prize winner—in a Nature “News and Views” article.
In the article, Wilczek contextualizes the team’s breakthrough. “Nuclear physics, and many major aspects of the physical world as we know it, hinges on the 0.14% difference in mass between neutrons and protons. Theoretically, that mass difference ought to be a calculable consequence of the quantum theory of the strong nuclear force (quantum chromodynamics; QCD) and the electromagnetic force (quantum electrodynamics; QED). But the required calculations are technically difficult and have long hovered out of reach. In a paper published in Science, [the team] report breakthrough progress on this problem,” he said in the Nature piece.
Some researchers, such as those led by Professors Wolfgang Schröder and Matthias Meinke from the Institute of Aerodynamics at RWTH Aachen University, benefit from the diversity and capabilities of GCS HPC resources.
The team focuses on understanding turbulence, one of the major unsolved problems in fluid dynamics. The team does not study turbulence in a purely theoretical way, though—it works closely with industry and experimentalists to understand turbulence in the context of making quieter, safer and, more energy efficient aircraft engines. The team is one of the largest users of supercomputing time at HLRS, and has benefited from significant amounts of computing time at JSC as well, allowing the team to get the “best of both worlds” by having access to two diverse computing architectures.
Through its various allocations, the team has been able to successfully simulate the fluid dynamics occurring on increasingly complex engine designs, beginning with helicopter engine dynamics, the simulating the radial fan blade on a typical aircraft, then moving to more advanced space launchers, also called “Chevron nozzles.”
Now in its 10th year, GCS has already begun implementing its strategy for the next decade of supercomputing excellence, or the “smart exascale” decade. The three GCS centres have all received funding for the next two rounds of supercomputers, and plan to not only further increase computing power, but also make their respective next-generation machines increasingly energy efficient. Faster supercomputers do not mean much without the ability to use them properly, though. To that end, GCS is further investing in training users and prospective users to make sure they spend less time porting codes or moving data and more time focused on their scientific research.
Leadership at the three GCS centres wants to continue delivering new solutions for their respective users, particularly in relation to disruptive technologies in the general HPC landscape. New computing architectures challenge traditional methods of doing simulation, and researchers need to be able to efficiently port their codes to make good use of a variety of compute architectures. Based on their decade of collaboration, the three GCS centres are excited about rising to this challenge by continuing to offer the variety of systems and integrated support structure built during the last 10 years.
By delivering on its promises, GCS and its funding agencies have secured the finances to ensure that GCS will continue to serve as Germany’s HPC leader and one of the world’s most powerful and innovative research institutions. In the next decade, GCS centres will continue to enable breakthrough science that benefits German stakeholders, the European scientific community, industry, and society in general. GCS wants to continue collaboration between other German and European HPC centres—such as its continued membership in the Partnership for Advanced Computing in Europe, or PRACE—and contribute to the best practices that help make Europe one of the strongest HPC regions in the world.
Computing technology will continue to rapidly evolve, though, challenging those working in HPC space. GCS staff will continue to work in the spirit of the organization’s namesake, Carl Friedrich Gauss, who was quoted, “It is not knowledge, but the act of learning, not possession, but the act of getting there, which grants the greatest enjoyment.”
contact: Eric Gedenk, gedenk[at]hlrs.de, eric.gedenk[at]gauss-centre.eu