A Bespoke Workflow Management System for Data-Driven Urgent HPC

Gordon P S Gibb, Nick Brown, Rupert W. Nash, Miguel Mendes, Santiago Monedero, Humberto Díaz Fidalgo, Joaquín Ramírez Cisneros, Adrián Cardil, Max Kontak

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper we present a workflow management system which permits the kinds of data-driven workflows required by urgent computing, namely where new data is integrated into the workflow as a disaster progresses in order refine the predictions as time goes on. This allows the workflow to adapt to new data at runtime, a capability that most workflow management systems do not possess. The workflow management system was developed for the EU-funded VESTEC project, which aims to fuse HPC with real-time data for supporting urgent decision making. We first describe an example workflow from the VESTEC project, and show why existing workflow technologies do not meet the needs of the project. We then go on to present the design of our Workflow Management System, describe how it is implemented into the VESTEC system, and provide an example of the workflow system in use for a test case.
Original languageEnglish
Title of host publication2020 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages10-20
Number of pages11
ISBN (Electronic)978-1-6654-2274-1
ISBN (Print)978-1-6654-2275-8
DOIs
Publication statusPublished - 30 Dec 2020
EventSecond international workshop on the use of HPC for urgent decision making - Virtual
Duration: 13 Nov 2020 → …
http://www.urgenthpc.com

Workshop

WorkshopSecond international workshop on the use of HPC for urgent decision making
Abbreviated titleUrgentHPC
Period13/11/20 → …
Internet address

Fingerprint Dive into the research topics of 'A Bespoke Workflow Management System for Data-Driven Urgent HPC'. Together they form a unique fingerprint.

Cite this