In this paper, we analyse teachers' judgments of students' written texts. We document how teachers use evidence in ways that depend both on their knowledge of the students and on the assessment framework they need to use. We analyse teachers' judgments by contrasting the structures of assessments made using teachers' normal classroom judgment processes with those made using an external set of ''benchmark'' standards. We show how the tension between demands for systemwide assessment validity and localised contextually sensitive site validity impacts on the richness and consistency of the judgment processes. We conclude that current understandings of teacher judgment processes that operate in everyday assessment practices generally fail to account for the complexity and dynamism of this routine classroom activity. Furthermore, we demonstrate that the methodology of judgment analysis, combined with think-aloud protocols, has the potential to shed light on the complexities associated with the operation of judgment in educational assessment.