OpenACC acceleration of the Nek5000 spectral element code

Stefano Markidis*, Jing Gong, Michael Schliephake, Erwin Laure, Alistair Hart, David Henty, Katherine Heisey, Paul Fischer

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

We present a case study of porting NekBone, a skeleton version of the Nek5000 code, to a parallel GPU-accelerated system. Nek5000 is a computational fluid dynamics code based on the spectral element method used for the simulation of incompressible flow. The original NekBone Fortran source code has been used as the base and enhanced by OpenACC directives. The profiling of NekBone provided an assessment of the suitability of the code for GPU systems, and indicated possible kernel optimizations. To port NekBone to GPU systems required little effort and a small number of additional lines of code (approximately one OpenACC directive per 1000 lines of code). The naïve implementation using OpenACC leads to little performance improvement: on a single node, from 16 Gflops obtained with the version without OpenACC, we reached 20 Gflops with the naïve OpenACC implementation. An optimized NekBone version leads to a 43 Gflop performance on a single node. In addition, we ported and optimized NekBone to parallel GPU systems, reaching a parallel efficiency of 79.9% on 1024 GPUs of the Titan XK7 supercomputer at the Oak Ridge National Laboratory.

Original languageEnglish
Pages (from-to)311-319
Number of pages9
JournalInternational Journal of High Performance Computing Applications
Issue number3
Publication statusPublished - 28 Aug 2015

Keywords / Materials (for Non-textual outputs)

  • Nek5000
  • OpenACC
  • optimization of NekBone with OpenACC
  • porting NekBone to GPU


Dive into the research topics of 'OpenACC acceleration of the Nek5000 spectral element code'. Together they form a unique fingerprint.

Cite this