Composing microservices and serverless for load resilience

Online services strive to maintain application responsiveness even when the traffic is unpredictable and fluctuating. Today’s online services are commonly deployed as graphs of microservices, each microservice packaged as one or more containers inside a virtual machines (VMs). While performant and affordable when the load is steady, VM-based deployments are known to be slow to scale when the load spikes, resulting in degraded performance for end-users of the service. To avoid such performance degradations, service providers can over-provision their deployments; however, such a strategy is costly and inefficient, leaving resources heavily under-utilized for extended periods of time. To address this challenge, we propose Hydra, a hybrid architecture that combines microservices with serverless computing. Hydra utilizes VMs to handle steady workloads cost-effectively and leverages serverless elasticity to absorb traffic spikes. When compared to an all-VM deployment with Kubernetes auto-scaling, Hydra achieves a 62.4% reduction in peak tail latency with a minimal 2.3% increase in cost.