Show simple item record

dc.contributor.authorLeigh, Catherine
dc.contributor.authorAlsibai, Omar
dc.contributor.authorHyndman, Rob J
dc.contributor.authorKandanaarachchi, Sevvandi
dc.contributor.authorKing, Olivia C
dc.contributor.authorMcGree, James M
dc.contributor.authorNeelamraju, Catherine
dc.contributor.authorStrauss, Jennifer
dc.contributor.authorTalagala, Priyanga Dilini
dc.contributor.authorTurner, Ryan DR
dc.contributor.authorMengersen, Kerrie
dc.contributor.authorPeterson, Erin E
dc.date.accessioned2019-06-07T01:34:46Z
dc.date.available2019-06-07T01:34:46Z
dc.date.issued2019
dc.identifier.issn0048-9697
dc.identifier.doi10.1016/j.scitotenv.2019.02.085
dc.identifier.urihttp://hdl.handle.net/10072/382992
dc.description.abstractMonitoring the water quality of rivers is increasingly conducted using automated in situ sensors, enabling timelier identification of unexpected values or trends. However, the data are confounded by anomalies caused by technical issues, for which the volume and velocity of data preclude manual detection. We present a framework for automated anomaly detection in high-frequency water-quality data from in situ sensors, using turbidity, conductivity and river level data collected from rivers flowing into the Great Barrier Reef. After identifying end-user needs and defining anomalies, we ranked anomaly importance and selected suitable detection methods. High priority anomalies included sudden isolated spikes and level shifts, most of which were classified correctly by regression-based methods such as autoregressive integrated moving average models. However, incorporation of multiple water-quality variables as covariates reduced performance due to complex relationships among variables. Classifications of drift and periods of anomalously low or high variability were more often correct when we applied mitigation, which replaces anomalous measurements with forecasts for further forecasting, but this inflated false positive rates. Feature-based methods also performed well on high priority anomalies and were similarly less proficient at detecting lower priority anomalies, resulting in high false negative rates. Unlike regression-based methods, however, all feature-based methods produced low false positive rates and have the benefit of not requiring training or optimization. Rule-based methods successfully detected a subset of lower priority anomalies, specifically impossible values and missing observations. We therefore suggest that a combination of methods will provide optimal performance in terms of correct anomaly detection, whilst minimizing false detection rates. Furthermore, our framework emphasizes the importance of communication between end-users and anomaly detection developers for optimal outcomes with respect to both detection performance and end-user application. To this end, our framework has high transferability to other types of high frequency time-series data and anomaly detection applications.
dc.description.peerreviewedYes
dc.languageEnglish
dc.publisherELSEVIER SCIENCE BV
dc.relation.ispartofpagefrom885
dc.relation.ispartofpageto898
dc.relation.ispartofjournalSCIENCE OF THE TOTAL ENVIRONMENT
dc.relation.ispartofvolume664
dc.subject.fieldofresearchMultidisciplinary
dc.subject.fieldofresearchcodeMD
dc.titleA framework for automated anomaly detection in high frequency water-quality data from in situ sensors
dc.typeJournal article
dc.type.descriptionC1 - Articles
dc.type.codeC - Journal Articles
dcterms.licensehttp://creativecommons.org/licenses/by-nc-nd/4.0/
gro.rights.copyright© 2019 Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Licence (http://creativecommons.org/licenses/by-nc-nd/4.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, providing that the work is properly cited.
gro.hasfulltextFull Text
gro.griffith.authorLeigh, Catherine
gro.griffith.authorKing, Olivia C.


Files in this item

This item appears in the following Collection(s)

  • Journal articles
    Contains articles published by Griffith authors in scholarly journals.

Show simple item record