PoCL-R: An open standard based heterogeneous offloading layer with server side scalability

Abstract

We propose PoCL-R, a novel computing runtime that makes remote compute devices available to the client via the cross-vendor heterogeneous Open Computing Language (OpenCL) API standard. For robustness in mobile use cases, intermittent connection loss is handled gracefully even if the device’s IP address changes on the way. A major benefit in comparison to the previous state-of-the-art OpenCL distribution layers is that PoCL-R minimizes network-induced latency by transferring data and signaling command completions between remote devices in a peer-to-peer fashion, outperforming the state-of-the-art baseline by a factor of up to 50x in synthetic benchmarks. PoCL-R’s streamlined TCP-based protocol has a command latency of only 60 microseconds on top of network round-trip latency in synthetic benchmarks. The application of PoCL-R to porting of compute-heavy applications to mobile devices is demonstrated with a smartphone-based Augmented Reality (AR) rendering case study where we measure up to 19x improvements to frame rate and 17x improvements to local energy consumption when using the proposed runtime to remote-offload a part of the AR rendering workload. Scalability across multiple GPU server nodes in real-world applications is shown using a computational fluid dynamics simulation, which scales with the number of servers at roughly 80% efficiency which is comparable to a Message-Passing Interface (MPI) port of the same simulation, but with the benefit of not needing a separate API for cross-server distribution.