Orchestrating data-centric workflows

Adam Barker*, Jon B. Weissman, Jano van Hemert

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

When orchestrating data-centric workflows as are commonly found in the sciences, centralised servers can become a bottleneck to the performance of a workflow; output from service invocations are normally transferred via a centralised orchestration engine, when they should be passed directly to where they are needed at the next service in the workflow. To address this performance bottleneck, this paper presents a lightweight hybrid workflow architecture and concrete API, based on a centralised control flow, distributed data flow model. Our architecture maintains the robustness and simplicity of centralised orchestration, but facilitates choreography by allowing services to exchange data directly with one another, reducing data that needs to be transferred through a centralised server. Furthermore our architecture is standards compliment, flexible and is a non-disruptive solution; service definitions do not have to be altered prior to enactment.
Original languageEnglish
Title of host publication2008 Eighth IEEE International Symposium on Cluster Computing and the Grid (CCGRID)
Pages210-217
Number of pages8
DOIs
Publication statusPublished - 2008
EventCCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid - Lyon, France
Duration: 19 May 200822 May 2008

Conference

ConferenceCCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid
Country/TerritoryFrance
CityLyon
Period19/05/0822/05/08

Keywords / Materials (for Non-textual outputs)

  • decentralised orchestration
  • systems architecture
  • web services
  • workflow optimisation

Fingerprint

Dive into the research topics of 'Orchestrating data-centric workflows'. Together they form a unique fingerprint.

Cite this