Detecting Technical Anomalies in High-Frequency Water-Quality Data Using Artificial Neural Networks
File version
Accepted Manuscript (AM)
Author(s)
Leigh, C
Liquet, B
Kermorvant, C
Peterson, E
Sous, D
Mengersen, K
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
Size
File type(s)
Location
License
Abstract
Anomaly detection (AD) in high-volume environmental data requires one to tackle a series of challenges associated with the typical low frequency of anomalous events, the broad-range of possible anomaly types, and local nonstationary environmental conditions, suggesting the need for flexible statistical methods that are able to cope with unbalanced high-volume data problems. Here, we aimed to detect anomalies caused by technical errors in water-quality (turbidity and conductivity) data collected by automated in situ sensors deployed in contrasting riverine and estuarine environments. We first applied a range of artificial neural networks that differed in both learning method and hyperparameter values, then calibrated models using a Bayesian multiobjective optimization procedure, and selected and evaluated the "best" model for each water-quality variable, environment, and anomaly type. We found that semi-supervised classification was better able to detect sudden spikes, sudden shifts, and small sudden spikes, whereas supervised classification had higher accuracy for predicting long-term anomalies associated with drifts and periods of otherwise unexplained high variability.
Journal Title
Environmental Science & Technology
Conference Title
Book Title
Edition
Volume
54
Issue
21
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
This work is covered by copyright. You must assume that re-use is limited to personal use and that permission from the copyright owner must be obtained for all other uses. If the document is available under a specified licence, refer to the licence for details of permitted re-use. If you believe that this work infringes copyright please make a copyright takedown request using the form at https://www.griffith.edu.au/copyright-matters.
Item Access Status
Note
Access the data
Related item(s)
Subject
Persistent link to this record
Citation
Rodriguez-Perez, J; Leigh, C; Liquet, B; Kermorvant, C; Peterson, E; Sous, D; Mengersen, K, Detecting Technical Anomalies in High-Frequency Water-Quality Data Using Artificial Neural Networks, Environmental Science & Technology, 2020, 54 (21), pp. 13719-13730