Deep Neural Networks (DNN)s have found plentiful opportunities to solve application challenges and issues within the scope of Internet of Things (IoT) networks, and further integration of these two fields continues to promise additional benefits. However, high-performance DNNs often have significant memory, computational power, and storage requirements. These requirements have been seen to continue to grow as cutting-edge networks become deeper and more complex in their pursuit of increasing what level of accuracy is possible. This trend in the field of deep learning presents a challenge to integrating DNNs in IoT environments, as IoT paradigms move towards incorporating processing elements closer to the edge to reduce transmission costs and latency.

Lightweight DNN model designs have been proposed as solutions to rectify these compatibility issues, but produce mixed results with such techniques often requiring either additional hardware resources moved to the edge or extensive modifications to already existing DNN models, increasing their development complexity and potentially incurring accuracy loss as a side effect of gaining performance improvements.

The goal of this thesis is to help researchers and developers combat the complexities and challenges of DNN model usage in IoT edge networks. This research examines one particularly promising lightweight model technique, early exit models, and investigates potential augmentations to its architecture and implementation to improve its performance in some of the major bottleneck areas lightweight models are facing. This thesis investigates improvements in three performance areas of DNN models use in IoT, including (i) exit accuracy, (ii) prediction uncertainty and noise robustness, and (iii) model deployment. For each improvement area, we focus on exploring alternative lightweight approaches to propose novel potential implementations of early exit models to solve the detrimental factors facing the challenge areas.

The main contributions of this thesis are:

Exit Accuracy: Early branch exits of lightweight models produce lower accuracy results than the model’s main exit, reducing the overall accuracy of early exiting models. To improve this we proposed the inclusion of knowledge distillation during the branch training process to transfer model knowledge to its earlier layers, this improves branch exit accuracy by over 5% and overall model accuracy by 2% without any additional computation or training.

Uncertainty and Noise: Branch exits perform poorly in the detection of out-of-distribution (OOD) inputs, decreasing model robustness against noisy IoT data, while requiring additional training time. To improve this, a new uncertainty estimation loss function, threshold calculation technique and training method are proposed to improve the accuracy of accepted predictions at all branch exits and training speed. This technique is shown to improve upon OOD detection rates from existing state-ofthe-art early exiting models by 7% and improve branch training speed 4fold.

Model Deployment: To reduce overall processing costs, early exit models suffer from the drawbacks of an increased memory footprint and additional processing for difficult-to-classify inputs. These limitations can potentially overburden IoT edge nodes and mean that the IoT data needs to be transferred to the cloud and thus incur network bandwidth costs. To solve this issue, we explore the concept of model splitting across both the edge and cloud within an IoT network and how this can enable partial deployment to the edge, reducing bandwidth usage. We propose a resource-aware optimisation function to determine the optimal splitting and branching point for a model to utilise available resources without incurring an overburden of the edge node.

In summary this thesis proposes novel implementations of early exiting models to build DNN models for IoT use that are both fast and effective in their role of providing processing for IoT applications. These techniques improve upon state-of-the-art early exiting architectures in areas of accuracy, robustness and IoT resource management at the edge.