In this paper, we introduce the CSPS framework for the hierarchical assessment of aquatic ecosystem models built on a range of metrics and characteristic signatures relevant to aquatic ecosystem condition. The framework is comprised of four levels: 0) conceptual validation; 1) comparison of simulated state variables with observations (‘state validation’); 2) comparison of fluxes with measured process rates (‘process validation’); and 3) assessment of system-level emergent properties, patterns and relationships (‘system validation’). Of these, only levels 0 and 1 are routinely undertaken at present. To highlight a diverse range of contexts relevant to the aquatic ecosystem modelling community, we present several case studies of improved validation approaches using the level 0–3 assessment hierarchy. We envision that the community–driven adoption of these metrics will lead to more rigorously assessed models, ultimately accelerating advances in model structure and function, and improved confidence in model predictions.