The EPCC OpenACC Benchmark Suite

OpenACC is a relatively new standard, first released in November 2011, which brings directives-based programming to accelerators such as NVidia or AMD GPUs, Intel MICs or other devices which can accelerate computational kernels. The use of accelerators as part of a heterogeneous multi-core system is one likely path towards exa-scale technologies. Programming models for such systems are highly likely to be directives-based as programmers will expect systems tools and compilers to make sensible choices about when and where to offload certain computational kernels dependent on a knowledge of the system.

In this work, we present a benchmark suite for OpenACC to compare systems and compilers. This suite provides three levels of benchmark ranging from low level data movement such as time taken to copy data to and from a device, through basic BLAS-type mathematical kernels such as DGEMM, ATAX and 2DCONV, to demonstration applications such as SEISMO, a 2D seismological simulation code. Each benchmark included is representative of operations commonly found in scientific computing.

To demonstrate the operation of the benchmark suite, we provide results obtained from the most recently released compilers from industry (CAPS, PGI, Cray) and academia (ULL). Our results show how a programmer can make an informed choice about which compiler would give best performance on their system. Alternatively, a programmer could use a single compiler but run the benchmarks on various systems to give an idea which system will best suit their application. This latter point is especially important where a choice of hardware is available, it may not be obvious which device is the best choice for a particular application without testing a range of devices.

We also envisage usage by compiler writers who can use our benchmarks to provide an independent reference source for the code they produce. Given the relative youth of the standard and implementations of that standard, it is important for compiler writers to be able to benchmark their compilers using externally provided codes.