Show simple item record

dc.contributor.authorTalagala, Priyanga Dilini
dc.contributor.authorHyndman, Rob J
dc.contributor.authorLeigh, Catherine
dc.contributor.authorMengersen, Kerrie
dc.contributor.authorSmith-Miles, Kate
dc.date.accessioned2019-12-11T23:25:11Z
dc.date.available2019-12-11T23:25:11Z
dc.date.issued2019
dc.identifier.issn0043-1397
dc.identifier.doi10.1029/2019WR024906
dc.identifier.urihttp://hdl.handle.net/10072/389682
dc.description.abstractOutliers due to technical errors in water‐quality data from in situ sensors can reduce data quality and have a direct impact on inference drawn from subsequent data analysis. However, outlier detection through manual monitoring is infeasible given the volume and velocity of data the sensors produce. Here we introduce an automated procedure, named oddwater, that provides early detection of outliers in water‐quality data from in situ sensors caused by technical issues. Our oddwater procedure is used to first identify the data features that differentiate outlying instances from typical behaviors. Then, statistical transformations are applied to make the outlying instances stand out in a transformed data space. Unsupervised outlier scoring techniques are applied to the transformed data space, and an approach based on extreme value theory is used to calculate a threshold for each potential outlier. Using two data sets obtained from in situ sensors in rivers flowing into the Great Barrier Reef lagoon, Australia, we show that oddwater successfully identifies outliers involving abrupt changes in turbidity, conductivity, and river level, including sudden spikes, sudden isolated drops, and level shifts, while maintaining very low false detection rates. We have implemented this oddwater procedure in the open source R package oddwater.
dc.description.peerreviewedYes
dc.languageEnglish
dc.language.isoeng
dc.publisherAmerican Geophysical Union (AGU)
dc.relation.ispartofjournalWater Resources Research
dc.subject.fieldofresearchPhysical Geography and Environmental Geoscience
dc.subject.fieldofresearchCivil Engineering
dc.subject.fieldofresearchEnvironmental Engineering
dc.subject.fieldofresearchcode0406
dc.subject.fieldofresearchcode0905
dc.subject.fieldofresearchcode0907
dc.titleA Feature-Based Procedure for Detecting Technical Outliers in Water-Quality Data From In Situ Sensors
dc.typeJournal article
dc.type.descriptionC1 - Articles
dcterms.bibliographicCitationTalagala, PD; Hyndman, RJ; Leigh, C; Mengersen, K; Smith-Miles, K, A Feature-Based Procedure for Detecting Technical Outliers in Water-Quality Data From In Situ Sensors, Water Resources Research, 2019
dc.date.updated2019-12-11T23:23:02Z
dc.description.versionVersion of Record (VoR)
gro.description.notepublicThis publication has been entered into Griffith Research Online as an Advanced Online Version.
gro.hasfulltextFull Text
gro.griffith.authorLeigh, Catherine


Files in this item

This item appears in the following Collection(s)

  • Journal articles
    Contains articles published by Griffith authors in scholarly journals.

Show simple item record