Supercomputing with MPI meets the Common Workflow Language standards: an experience report

Rupert W. Nash, Nick Brown, Michael R Crusoe, Max Kontak

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Use of standards-based workflows is still somewhat unusual by high-performance computing users. In this paper we describe the experience of using the Common Workflow Language (CWL) standards to describe the execution, in parallel, of MPI-parallelised applications. In particular, we motivate and describe the simple extension to the specification which was required, as well as our implementation of this within the CWL reference runner. We discuss some of the unexpected benefits, such as simple use of HPC-oriented performance measurement tools, and CWL software requirements interfacing with HPC module systems. We close with a request for comment from the community on how these features could be adopted within versions of the CWL standards.
Original languageEnglish
Title of host publication2020 IEEE/ACM Workflows in Support of Large-Scale Science (WORKS)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages17-24
Number of pages8
ISBN (Electronic)978-0-7381-1040-0
ISBN (Print)978-0-7381-1041-7
DOIs
Publication statusPublished - 4 Jan 2021
Event15th Workshop on Workflows in Support of Large-Scale Science - Virtual
Duration: 11 Nov 202011 Nov 2020
https://works-workshop.org/

Workshop

Workshop15th Workshop on Workflows in Support of Large-Scale Science
Abbreviated titleWORKS 2020
CityVirtual
Period11/11/2011/11/20
Internet address

Fingerprint Dive into the research topics of 'Supercomputing with MPI meets the Common Workflow Language standards: an experience report'. Together they form a unique fingerprint.

Cite this