Abstract / Description of output
As more data-intensive computing applications are executed on high performance computing clusters, resource contention on the shared storage system attached to the clusters becomes significant. The contention might cause I/O performance degradation and spoil performance improvement of coordinated parallel I/O by the MPI-IO implementation. In order to solve this problem, an advanced reservation approach where storage resources are managed based on the reservations to satisfy the I/O performance requirements, has been proposed. In this paper, we apply the concept of reserved data access to MPI-IO, in particular to Two-Phase collective I/O which is primarily used for I/O aggregation in non-contiguous access by MPI applications. We developed a prototype by using Dynamic-CoMPI which supports further improvement of Two-Phase I/O by using a locality aware strategy, and Papio which is a parallel storage system providing performance reservation functionality. After describing our prototype design and implementation, we show leverage of the concept by comparing our implementation with other existing MPI-IO implementations backed by OrangeFS and Lustre. The evaluation experiment confirms that the optimization benefit of Two-Phase I/O can be preserved by our approach, under the resource contention situation.
Original language | English |
---|---|
Title of host publication | Cluster Computing (CLUSTER), 2013 IEEE International Conference on |
Publisher | Institute of Electrical and Electronics Engineers |
Pages | 1-5 |
Number of pages | 5 |
DOIs | |
Publication status | Published - Sept 2013 |
Keywords / Materials (for Non-textual outputs)
- input-output programs
- message passing
- parallel processing
- resource allocation
- storage management
- Dynamic-CoMPI
- IO aggregation
- IO performance requirements
- Lustre
- MPI collective input-output
- MPI-IO implementation
- OrangeFS
- Papio system
- advanced reservation approach
- coordinated parallel IO
- data-intensive computing applications
- high performance computing clusters
- locality aware strategy
- performance guarantees
- resource contention
- shared storage systems
- storage resources
- two-phase collective IO
- Computational modeling
- Quality of service