Decoupling data computation from processing to support high performance data analytics

Activity: Academic talk or presentation typesInvited talk


There are numerous HPC codes that not only perform computation, but also analyse their values to generate higher level information from this raw data. Atmospheric science is one of these domains and traditionally computation has been paused whilst higher level diagnostic values are begot from the raw prognostic fields. In our approach we share the cores of a processor between simulation and analytics/IO, typically one core of the processor performing the analytics and servicing the remainder simulation cores which asynchronously fire and forget their raw data over to be processed. By decoupling these two aspects the theory is that they can both progress fairly independently and this can hide the heavy communication and IO costs of analytics from the simulation side of things. In our work we have realised that a task-based paradigm, but one that is explicitly aware of different ranks, is advantageous in structuring the code and handling the non-deterministic nature of data arrival and processing.
Period19 Apr 2018
Event titleEuropean Exascale Applications Workshop
Event typeConference
LocationEdinburgh, United KingdomShow on map
Degree of RecognitionInternational


  • Event Driven Asynchronous Task
  • EDAT
  • In-situ
  • In-situ data analytics