MaxPair: Enhance OpenCL Concurrent Kernel Execution by Weighted Maximum Matching

Executing multiple OpenCL kernels on the same GPU concurrently
is a promising method for improving hardware utilisation and
system performance. Schemes of scheduling impact the resulting
performance significantly by selecting different kernels to run together on the same GPU. Existing approaches use either execution time or relative speedup of kernels as a guide to group and map them to the device. However, these simple methods work on the cost of providing suboptimal performance.
In this paper, we propose a graph-based algorithm to schedule
co-run kernel in pairs to optimise the system performance. Target
workloads are represented by a graph, in which vertices stand for distinct kernels while edges between two vertices represent the corresponding two kernels co-execution can deliver a better
performance than run them one after another. Edges are weighted
to provide information of performance gain from co-execution. Our algorithm works in the way of finding out the maximum weighted matching of the graph. By maximising the accumulated weights, our algorithm improves performance significantly
comparing to other approaches.