Assessing Industrial Control System Attack Datasets for Intrusion Detection

Abstract

With the rapid development of networks and computers, industrial control systems (ICS) have become more interconnected. Many ICS are allowed remote interactions through the Internet. This increases the security risks of being attacked. If critical infrastructure ICS are attacked, the consequences could be catastrophic. To protect the ICS, the anomaly-based network intrusion detection systems (ABNIDS) are used to detect novel cyber-attacks by learning both normal and abnormal network behaviours. The quality of the attack dataset directly influences the accuracy of the ABNIDS. Therefore, it is important to assess the quality of the attack datasets used to design and develop ABNIDS. To fulfil this goal, this paper provides assessment criteria for evaluating ICS attack datasets. These new assessment criteria demonstrate the various requirements of the dataset and analyse the effectiveness of the dataset in depth. Three existing ICS attack datasets for the DNP3, S7comm and Modbus protocols are assessed using these criteria. We find that there is a range of dataset creation techniques and levels of quality with no dataset that meets the ideal criteria. Since no existing work discusses assessment criteria for ICS attack datasets, this paper would be helpful to evaluate and improve the ICS attack datasets.