Resource requirement specification for novel data-aware and workflow-enabled HPC job schedulers

Manos Farsarakis; Iakovos Panourgias; William Jackson; Juan F. R. Herrera; Michele Weiland; Mark Parsons

Resource requirement specification for novel data-aware and workflow-enabled HPC job schedulers

Manos Farsarakis, Iakovos Panourgias, William Jackson, Juan F. R. Herrera, Michele Weiland, Mark Parsons

Research output: Contribution to conference › Other › peer-review

Abstract / Description of output

Technological advancements in computer architectures and the ever-increasing scope of what may be described as a hybrid HPC architecture have not been followed by relevant changes in the way jobs are described to HPC job schedulers. In this WiP, we aim to introduce an augmented job resource request (JRR) specification to be adapted by existing job scheduler implementations and used to more accurately describe the resource requirements of a job to HPC schedulers. The ultimate aim of this work is to both improve the performance of individual applications by improving the utilization of novel resources as they become available, as well as to enable the more efficient scheduling of jobs and workflows on future HPC systems.

Original language	English
Number of pages	1
Publication status	Published - 2017
Event	Supercomputing 2017 - Denver, United States Duration: 13 Nov 2017 → 17 Nov 2017 http://sc17.supercomputing.org/

Conference

Conference	Supercomputing 2017
Abbreviated title	SC17
Country/Territory	United States
City	Denver
Period	13/11/17 → 17/11/17
Internet address	http://sc17.supercomputing.org/

Keywords / Materials (for Non-textual outputs)

Storage class memory
Workflows
Scalability
NEXTGenIO
Resource allocation
SLURM
Portable Batch Scheduler
HPC
Systemware

Access to Document

http://www.pdsw.org/pdsw-discs17/wips/farsarakis-wip-pdsw-discs17.pdf

Cite this

@conference{eb036c2f2f8c43f4b61abc829d095436,

title = "Resource requirement specification for novel data-aware and workflow-enabled HPC job schedulers",

abstract = "Technological advancements in computer architectures and the ever-increasing scope of what may be described as a hybrid HPC architecture have not been followed by relevant changes in the way jobs are described to HPC job schedulers. In this WiP, we aim to introduce an augmented job resource request (JRR) specification to be adapted by existing job scheduler implementations and used to more accurately describe the resource requirements of a job to HPC schedulers. The ultimate aim of this work is to both improve the performance of individual applications by improving the utilization of novel resources as they become available, as well as to enable the more efficient scheduling of jobs and workflows on future HPC systems.",

keywords = "Storage class memory, Workflows, Scalability, NEXTGenIO, Resource allocation, SLURM, Portable Batch Scheduler, HPC, Systemware",

author = "Manos Farsarakis and Iakovos Panourgias and William Jackson and Herrera, {Juan F. R.} and Michele Weiland and Mark Parsons",

year = "2017",

language = "English",

note = "Supercomputing 2017, SC17 ; Conference date: 13-11-2017 Through 17-11-2017",

url = "http://sc17.supercomputing.org/",

}

TY - CONF

T1 - Resource requirement specification for novel data-aware and workflow-enabled HPC job schedulers

AU - Farsarakis, Manos

AU - Panourgias, Iakovos

AU - Jackson, William

AU - Herrera, Juan F. R.

AU - Weiland, Michele

AU - Parsons, Mark

PY - 2017

Y1 - 2017

N2 - Technological advancements in computer architectures and the ever-increasing scope of what may be described as a hybrid HPC architecture have not been followed by relevant changes in the way jobs are described to HPC job schedulers. In this WiP, we aim to introduce an augmented job resource request (JRR) specification to be adapted by existing job scheduler implementations and used to more accurately describe the resource requirements of a job to HPC schedulers. The ultimate aim of this work is to both improve the performance of individual applications by improving the utilization of novel resources as they become available, as well as to enable the more efficient scheduling of jobs and workflows on future HPC systems.

AB - Technological advancements in computer architectures and the ever-increasing scope of what may be described as a hybrid HPC architecture have not been followed by relevant changes in the way jobs are described to HPC job schedulers. In this WiP, we aim to introduce an augmented job resource request (JRR) specification to be adapted by existing job scheduler implementations and used to more accurately describe the resource requirements of a job to HPC schedulers. The ultimate aim of this work is to both improve the performance of individual applications by improving the utilization of novel resources as they become available, as well as to enable the more efficient scheduling of jobs and workflows on future HPC systems.

KW - Storage class memory

KW - Workflows

KW - Scalability

KW - NEXTGenIO

KW - Resource allocation

KW - SLURM

KW - Portable Batch Scheduler

KW - HPC

KW - Systemware

UR - http://www.pdsw.org/pdsw-discs17/index.shtml

M3 - Other

T2 - Supercomputing 2017

Y2 - 13 November 2017 through 17 November 2017

ER -

Resource requirement specification for novel data-aware and workflow-enabled HPC job schedulers

Abstract / Description of output

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

NEXTGenIO: Next Generation I/O for Exascale

Cite this

Resource requirement specification for novel data-aware and workflow-enabled HPC job schedulers

Abstract / Description of output

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Projects

NEXTGenIO: Next Generation I/O for Exascale

Cite this