Exploiting Time-Malleability in Cloud-based Batch Processing Systems

Luo Mai, Evangelia Kalyvianaki, Paolo Costa

Research output: Contribution to conferencePaperpeer-review

Abstract

Existing cloud provisioning schemes allocate resources to batch processing systems at deployment time and only change this allocation at run-time due to unexpected events such as server failures.

We observe that MapReduce-like jobs are timemalleable, i.e., at runtime it is possible to dynamically vary the number of resources allocated to a job and, hence, its completion time.

In this paper, we propose a novel approach based on time-malleability to opportunistically update job resources in order to increase overall utilization and revenue. To set the right incentives for both providers and tenants, we introduce a novel pricing model that charges tenants according to job completion times. Using this model, we formulate an optimization problem for revenue maximization.

Preliminary results show that compared to today’s practices our solution can increase revenue by up to 69.7% and can accept up to 57% more jobs.
Original languageEnglish
Number of pages6
Publication statusPublished - 13 Nov 2013
Event7th Workshop on Large-Scale Distributed Systems and Middleware - Farmington, United States
Duration: 2 Nov 20133 Nov 2013
http://sigops.org/s/conferences/sosp/2013/ladis.html

Workshop

Workshop7th Workshop on Large-Scale Distributed Systems and Middleware
Abbreviated titleLADIS '13
CountryUnited States
CityFarmington
Period2/11/133/11/13
Internet address

Fingerprint Dive into the research topics of 'Exploiting Time-Malleability in Cloud-based Batch Processing Systems'. Together they form a unique fingerprint.

Cite this