Edge Offloading of Low-Latency Computer Vision Tasks in Challenging Network Conditions

Abstract

Recent advances in fast networking technology and Artificial Intelligence (Al) have enabled a broad range of machine-to-machine communication applications in smart factories, search and rescue, agriculture, autonomous vehicles, and other domains. Many of these applications rely on performing real-time Computer Vision (CV) tasks, such as semantic segmentation or object detection, that are typically realized by computationally intensive Neural Networks (NN), which makes them challeng­ing to perform on resource-constrained lightweight devices. Offloading the NN inference to a remote server enables smaller devices to perform more complex tasks but requires fast and robust communication.

Commonly, lossy image and video compression has been used to reduce the trans­mitted data rate and thus the time and energy required to transmit the data, at the cost of additional computation and reduced accuracy of the performed task. Modern codecs achieve high compression efficiency at the cost of additional computational complexity compared to previous generations. While reducing the complexity of such codecs is an active research area, computationally restricted devices can still lack the resources to encode the input with sufficiently low latency. Thus, there is a need to search for even lower-complexity compression algorithms specifically targeting Al inference. This search can be aided by exploiting the fact that the video is consumed by an algorithm, not a human, and such can be tolerable to certain kinds of artifacts that would be detrimental for a human. Furthermore, as the encoded data is transmitted via a wireless network channel, the channel quality plays a major role in the reconstructed signal quality and thus the performance of the whole system. Moreover, the network conditions can change over time, and the edge offloading system must be able to adapt to them.

This thesis addresses the above problems on multiple fronts. First, existing image compression algorithms are studied in terms of their encoding latency and the im­pact of their artifacts on CV task quality. In particular, a proposed implementation of an Adaptive Scalable Texture Compression (ASTC) encoder achieves 2.3 times faster encoding than a state-of-the-art Joint Photographic Experts Group (JPEG) encoding library on a mobile device. Second, a demonstrator application is developed to demonstrate the latency and power savings of lightweight compression in a practical system considering different networking scenarios. An arbiter dynamically deciding between local and remote execution and the optimal compression strategy based on periodically monitored system metrics is proposed as a part of this system. Measurements using commercially available SG and Wi-Fi 6 networks demonstrate the ability of the system to prioritize fast encoding at the cost of worse accuracy in the case of network connection degradation, but retain maximum accuracy at higher bandwidth if the connection quality is good enough. Third, by considering the wireless network channel quality, a new joint source-channel coding based on Linear Coding and Transmission (LCT) is developed specifically for improved CV task accuracy. For the same accuracy, the proposed scheme transmits on average 15-28% fewer symbols than a baseline LCT at a negligible computational overhead. This improvement is further validated using realistic SG channel simulations.

By reducing the cost of offloading CV applications, the contributions of this thesis facilitate deploying more advanced intelligence on less powerful devices. Reducing the offloading costs allows sharing and re-use of computing resources at the server, which would otherwise have to be duplicated at each client device. Thus, the results of this thesis can help reduce energy and materials consumed by intelligent devices, especially as their number is expected to continue growing in the near future.