DeepAuto: A first step towards formal verification of deep learning systems

Abstract

Deep Learning (DL) offers a data-driven programming paradigm in which Deep Neural Networks (DNNs) can be constructed through a set of training data. It has been widely adopted in many real-world applications. However, many studies have shown that DL systems suffer from adversarial attacks, especially when they are applied to security- and safety-critical domains. Given that formal verification has proved a great success in many areas such as software engineering, using it to achieve a high-level security assurance in DL systems is considered promising. In this paper, we design and implement DeepAuto which makes the significant bridge between automata and DNNs. With the aid of DeepAuto, we demonstrate how DNNs can be modeled as automata and be verified formally in the widely used model checker UPPAAL. The potential usefulness of DeepAuto shows the connection between DNNs and automata and provides a solution for the construction of more trustworthy DL systems.