Scientific Workflows: Moving Across Paradigms

Chee Sun Liew, Malcolm Atkinson, Michelle Galea, Tan Fong Ang, Paul Martin, Jano van Hemert

Research output: Contribution to journalArticlepeer-review

Abstract

Modern scientific collaborations have opened up the opportunity to solve complex problems that require both multidisciplinary expertise and large-scale computational experiments. These experiments typically comprise a sequence of processing steps that need to be executed on selected computing platforms. Execution poses a challenge however due to a) the complexity and diversity of applications, b) the diversity of analysis goals, c) the heterogeneity of computing platforms, and d) the volume and distribution of data. A common strategy to make these in silico experiments more manageable is to model them as workflows, and to use a workflow management system to organise their execution. This article looks at the overall challenge posed by a new order of scientific experiments and the systems they need to be run on, and examines how this challenge can be addressed by workflows and workflow management systems. It proposes a taxonomy workflow management system (WMS) characteristics, including aspects previously overlooked. This frames a review of prevalent WMS used by the scientific community, elucidates their evolution to handle the challenges arising with the emergence of the ‘fourth paradigm’ and identifies research needed to maintain progress in this area.
Original languageEnglish
Article number66
Number of pages39
JournalACM Computing Surveys
Volume49
Issue number4
Early online date1 Dec 2016
DOIs
Publication statusPublished - 1 Feb 2017

Keywords

  • parallel architectures
  • software engineering
  • distributed architectures

Fingerprint

Dive into the research topics of 'Scientific Workflows: Moving Across Paradigms'. Together they form a unique fingerprint.

Cite this