Forecasting accurate water quality is very important in aquaculture, environment monitoring, and many other applications. Many internal and external factors influence water quality. Therefore, water quality parameters exhibit complex time series characteristics. Consequently, long-range accurate prediction of water quality parameters suffers from poor propagation of information from past timepoints to further future timepoints. Moreover, to synchronise the prediction model with the changes in the time series characteristics, periodic retraining of the prediction model is required and such retraining is to be done on resource-restricted computation devices. In this work, we present a low-cost training approach to improve long-range multi-step water quality prediction. We train a short-range predictor to save training effort. Then, we strive to achieve and/or improve long-range prediction using multi-step iterative ensembling during inference. Experimental results on 9 water quality datasets demonstrate that the proposed method achieves significantly lower error than the existing state-of-the-art approaches. Our approach significantly outperforms the existing approaches in several standard metrics, even in the case of future timepoints at long distances.