Accuracy in the scoring of writing: Studies of reliability and validity using a New Zealand writing assessment system

No Thumbnail Available
File version
Author(s)
T.L. Brown, Gavin
Glasswell, Kath
Harland, Don
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
2004
Size
File type(s)
Location
License
Abstract

Accuracy in the scoring of writing is critical if standardized tasks are to be used in a national assessment scheme. Three approaches to establishing accuracy (i.e., consensus, consistency, and measurement) exist and commonly large-scale assessment programs of primary school writing demonstrate adjacent agreement consensus rates of between 80% and 100%, and consistency and measurement coefficients ranging between .70 and .80, .60 and .80, respectively. A New Zealand educational assessment project has developed a set of writing assessment rubrics that contain curriculum based rating scales for six purposes of writing each with its own bank of writing prompts for use by classroom teachers. Standardization of the rating scales and prompts was conducted with representative samples of students, aged 10-13. This article describes two studies that established the validity of the scoring system for use in New Zealand classrooms. Adjacent agreement consensus fell between 70% and 90%, while consistency and measurement correlations fell in the range .70-.80 in both studies. This consistency with international standards was sufficiently robust to provide confidence in the underlying norms provided by the asTTle assessment tool. Relatively low levels of training were required by teachers to reach this degree of accuracy. The accuracy of scoring gives government and teachers confidence in the validity of the project's rating scales and suggests that classroom teachers will be able to generate accurate scores upon which instructional decisions can be based.

Journal Title

Assessing Writing

Conference Title
Book Title
Edition
Volume

9

Issue

2

Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
Item Access Status
Note
Access the data
Related item(s)
Subject

Education Assessment and Evaluation

Specialist Studies in Education

Linguistics

Persistent link to this record
Citation
Collections