Abstract / Description of output
Serverless computing is a popular software deployment model for the cloud, in which applications are designed as a collection of stateless tasks. Developers are charged for the CPU time and memory footprint during the execution of each serverless function, which incentivizes them to reduce both runtime and memory usage. As a result, functions tend to be short (often on the order of a few milliseconds) and compact (128–256 MB). Cloud providers can pack thousands of such functions on a server, resulting in frequent context switches and a tremendous degree of interleaving. As a result, when a given memory-resident function is re-invoked, it commonly finds its on-chip microarchitectural state completely
cold due to thrashing by other functions — a phenomenon termed lukewarm invocation.
Our analysis shows that the cold microarchitectural state due to lukewarm invocations is highly detrimental to performance, which corroborates prior work. The main source of performance degradation is the front-end, composed of instruction delivery, branch identification via the BTB and the conditional branch prediction. State-of-the-art front-end prefetchers show only limited effectiveness on lukewarm invocations, falling considerably short of an ideal front-end. We demonstrate that the reason for this is the cold microarchitectural state of the branch identification and prediction units. In response, we introduce Ignite, a comprehensive restoration mechanism for front-end microarchitectural state targeting instructions, BTB and branch predictor via unified metadata. Ignite records an invocation’s control flow graph in compressed format and uses that to restore the front-end structures the next time the function is invoked. Ignite outperforms state-of-the-art front-end prefetchers, improving performance by an average of 43% by significantly reducing instruction, BTB and branch predictor MPKI.
cold due to thrashing by other functions — a phenomenon termed lukewarm invocation.
Our analysis shows that the cold microarchitectural state due to lukewarm invocations is highly detrimental to performance, which corroborates prior work. The main source of performance degradation is the front-end, composed of instruction delivery, branch identification via the BTB and the conditional branch prediction. State-of-the-art front-end prefetchers show only limited effectiveness on lukewarm invocations, falling considerably short of an ideal front-end. We demonstrate that the reason for this is the cold microarchitectural state of the branch identification and prediction units. In response, we introduce Ignite, a comprehensive restoration mechanism for front-end microarchitectural state targeting instructions, BTB and branch predictor via unified metadata. Ignite records an invocation’s control flow graph in compressed format and uses that to restore the front-end structures the next time the function is invoked. Ignite outperforms state-of-the-art front-end prefetchers, improving performance by an average of 43% by significantly reducing instruction, BTB and branch predictor MPKI.
Original language | English |
---|---|
Title of host publication | 2023 56th IEEE/ACM International Symposium on Microarchitecture (MICRO) |
Publisher | Association for Computing Machinery (ACM) |
Pages | 254-267 |
Number of pages | 14 |
ISBN (Electronic) | 9798400703294 |
DOIs | |
Publication status | Published - 8 Dec 2023 |
Event | 56th IEEE/ACM International Symposium on Microarchitecture - Toronto, Canada Duration: 28 Oct 2023 → 1 Nov 2023 https://microarch.org/micro56/ |
Conference
Conference | 56th IEEE/ACM International Symposium on Microarchitecture |
---|---|
Abbreviated title | MICRO 2023 |
Country/Territory | Canada |
City | Toronto |
Period | 28/10/23 → 1/11/23 |
Internet address |
Keywords / Materials (for Non-textual outputs)
- Microarchitecture
- instruction delivery
- front-end prefetching
- serverless