AI models on constrained embedded devices

Abstract

In recent years, the most common way to use data in the Internet of Things is to parse and compute in the cloud with artificial intelligence. However, the amount of data generated grows each day and communication between the cloud and edge devices causes latency, power consumption, and possible security risks. With the emergence of tiny machine learning, computing near or on edge devices becomes more feasible and provides fixes for the problems of latency, power consumption, and security.

This thesis explores computing on the extreme edge with inference on the device without any connection to the Internet. More specifically, methods to make edge machine learning possible are evaluated and tested on the popular Raspberry Pi Pico W development board.

The evaluation explores tiny machine learning frameworks and embedded machine learning in an educational context. From a pool of candidates, two tiny machine learning frameworks, called Edge Impulse and TensorFlow Lite Micro, are chosen. They are used to build and use models for a simple application that runs inferences on real-time sensor data. Quantitative metrics such as the size and accuracy of the model are used to compare the results, and qualitative metrics such as the difficulty to use the framework are used to compare the user experience.

The findings vary, and both frameworks have their own use cases. Edge Impulse is found to be more beginner-friendly and does not require knowledge of machine learning. On the other hand, TensorFlow Lite Micro requires a good understanding of machine learning and might be more suitable for users interested in the actual methods of creating embedded suitable machine learning.

These findings can be used to determine which tiny machine learning framework to use for a specific use case.