Benchmarking machine learning models at the edge
Haider, Md Masud (2024)
2024
Tietojenkäsittelyopin maisteriohjelma - Master's Programme in Computer Science
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2024-12-08
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-2024112610493
https://urn.fi/URN:NBN:fi:tuni-2024112610493
Tiivistelmä
Recently the use of resource-constrained edge devices to perform machine learning tasks has gained substantial research attention. The need to perform such tasks arises from the vast amounts of sensors, and IoT and smart devices that generate large swaths of data at an enormous rate. To process such data in a real-time manner, cloud computing has proven to be suboptimal.
As an alternative, edge devices paired with ML models can execute demanding tasks in diverse environments. This study explored a range of machine learning models coupled with several low and high-end edge devices to test their suitability for deployment in an edge environment. Unlike previous works, this experiment takes into account the initial heavy lifting that machine learning models and devices go through before performing the actual task of classification or detection. To systematically benchmark and gather results, this study divides the ML pipeline into initial and execution phases while thoroughly measuring each phase’s time and resource usage. This type of rigorous performance metric analysis enables decision-making about which machine learning model and devices have better suitability for latency and speed. Moreover, to add a device in the suit of existing ones and replicate the benchmark workflow, this study has automated the benchmark with the Ansible automation tool.
The thesis research presents its findings from different devices. The division of phases allows for a clear understanding of the bottlenecks experienced by the devices and their capabilities. Moreover, an underlying crucial detail such as the warmup was revealed by scrutinizing the performances of the devices from the very beginning.
The thesis concludes with performance highlights and limitations of the benchmark, identifying possible future research directions that can be developed further.
As an alternative, edge devices paired with ML models can execute demanding tasks in diverse environments. This study explored a range of machine learning models coupled with several low and high-end edge devices to test their suitability for deployment in an edge environment. Unlike previous works, this experiment takes into account the initial heavy lifting that machine learning models and devices go through before performing the actual task of classification or detection. To systematically benchmark and gather results, this study divides the ML pipeline into initial and execution phases while thoroughly measuring each phase’s time and resource usage. This type of rigorous performance metric analysis enables decision-making about which machine learning model and devices have better suitability for latency and speed. Moreover, to add a device in the suit of existing ones and replicate the benchmark workflow, this study has automated the benchmark with the Ansible automation tool.
The thesis research presents its findings from different devices. The division of phases allows for a clear understanding of the bottlenecks experienced by the devices and their capabilities. Moreover, an underlying crucial detail such as the warmup was revealed by scrutinizing the performances of the devices from the very beginning.
The thesis concludes with performance highlights and limitations of the benchmark, identifying possible future research directions that can be developed further.