MPI collective I/O based on advanced reservations to obtain performance guarantees from shared storage systems

Y. Tanimura, R. Filgueira, I. Kojima, M. Atkinson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As more data-intensive computing applications are executed on high performance computing clusters, resource contention on the shared storage system attached to the clusters becomes significant. The contention might cause I/O performance degradation and spoil performance improvement of coordinated parallel I/O by the MPI-IO implementation. In order to solve this problem, an advanced reservation approach where storage resources are managed based on the reservations to satisfy the I/O performance requirements, has been proposed. In this paper, we apply the concept of reserved data access to MPI-IO, in particular to Two-Phase collective I/O which is primarily used for I/O aggregation in non-contiguous access by MPI applications. We developed a prototype by using Dynamic-CoMPI which supports further improvement of Two-Phase I/O by using a locality aware strategy, and Papio which is a parallel storage system providing performance reservation functionality. After describing our prototype design and implementation, we show leverage of the concept by comparing our implementation with other existing MPI-IO implementations backed by OrangeFS and Lustre. The evaluation experiment confirms that the optimization benefit of Two-Phase I/O can be preserved by our approach, under the resource contention situation.
Original languageEnglish
Title of host publicationCluster Computing (CLUSTER), 2013 IEEE International Conference on
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages1-5
Number of pages5
DOIs
Publication statusPublished - Sep 2013

Keywords

  • input-output programs
  • message passing
  • parallel processing
  • resource allocation
  • storage management
  • Dynamic-CoMPI
  • IO aggregation
  • IO performance requirements
  • Lustre
  • MPI collective input-output
  • MPI-IO implementation
  • OrangeFS
  • Papio system
  • advanced reservation approach
  • coordinated parallel IO
  • data-intensive computing applications
  • high performance computing clusters
  • locality aware strategy
  • performance guarantees
  • resource contention
  • shared storage systems
  • storage resources
  • two-phase collective IO
  • Computational modeling
  • Quality of service

Fingerprint Dive into the research topics of 'MPI collective I/O based on advanced reservations to obtain performance guarantees from shared storage systems'. Together they form a unique fingerprint.

Cite this