Reconstructing missing and anomalous data collected from high-frequency in-situ sensors in fresh waters

Loading...
Thumbnail Image
File version

Version of Record (VoR)

Author(s)
Kermorvant, C
Liquet, B
Litt, G
Jones, JB
Mengersen, K
Peterson, EE
Hyndman, RJ
Leigh, C
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
2021
Size
File type(s)
Location
Abstract

In situ sensors that collect high-frequency data are used increasingly to monitor aquatic environments. These sensors are prone to technical errors, resulting in unrecorded observations and/or anomalous values that are subsequently removed and create gaps in time series data. We present a framework based on generalized additive and auto-regressive models to recover these missing data. To mimic sporadically missing (i) single observations and (ii) periods of contiguous observations, we randomly removed (i) point data and (ii) day-and week-long sequences of data from a two-year time series of nitrate concentration data collected from Arikaree River, USA, where synoptically collected water temperature, turbidity, conductance, elevation, and dissolved oxygen data were available. In 72% of cases with missing point data, predicted values were within the sensor precision interval of the original value, although predictive ability declined when sequences of missing data occurred. Precision also depended on the availability of other water quality covariates. When covariates were available, even a sudden, event-based peak in nitrate concentration was reconstructed well. By providing a promising method for accurate prediction of missing data, the utility and confidence in summary statistics and statistical trends will increase, thereby assisting the effective monitoring and management of fresh waters and other at-risk ecosystems.

Journal Title

International Journal of Environmental Research and Public Health

Conference Title
Book Title
Edition
Volume

18

Issue

23

Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Item Access Status
Note
Access the data
Related item(s)
Subject

Environmental sciences

anomaly correction

generalised additive model (GAM)

missing data reconstruction

remote sensing

water quality

Persistent link to this record
Citation

Kermorvant, C; Liquet, B; Litt, G; Jones, JB; Mengersen, K; Peterson, EE; Hyndman, RJ; Leigh, C, Reconstructing missing and anomalous data collected from high-frequency in-situ sensors in fresh waters, International Journal of Environmental Research and Public Health, 2021, 18 (23), pp. 12803

Collections