Towards more efficient assessments – increasing information from objective examinations

View/ Open
Author(s)
Dugdale, Alan
Griffith University Author(s)
Year published
2013
Metadata
Show full item recordAbstract
Objective examination questions are widely used to assess students' knowledge, but the standard MCQ with a stem followed by 4 - 6 possible answers one of which is chosen as correct is very inefficient. Sim-ple changes to the format can treble the information gained. Information theory is the tool for assessing the information content of electronic and other communications. The "bit" is the unit of information and equals on true/false choice. I have applied basic information theory to objective questions. The standard MCQ with a stem, a choice of 4 possible answers (mark one answer true) and an expected correct answer rate ...
View more >Objective examination questions are widely used to assess students' knowledge, but the standard MCQ with a stem followed by 4 - 6 possible answers one of which is chosen as correct is very inefficient. Sim-ple changes to the format can treble the information gained. Information theory is the tool for assessing the information content of electronic and other communications. The "bit" is the unit of information and equals on true/false choice. I have applied basic information theory to objective questions. The standard MCQ with a stem, a choice of 4 possible answers (mark one answer true) and an expected correct answer rate of 70% yields 1.36 bits. A MTFQ with 4 choices where the student must answer true/false to every possible answer gives 3.52 bits of information. By adding a "don't know" option the same MTFQ gives 4.72 bits of information, 350% of the standard MCQ. Thirty MTFQ with don't know give the same in-formation about students' knowledge as 100 standard MCQs. The effort needed to set, sit and mark is the same for both. Small changes to the format of objective questions give large gains in efficiency. We should balance these gains against possible disadvantages.
View less >
View more >Objective examination questions are widely used to assess students' knowledge, but the standard MCQ with a stem followed by 4 - 6 possible answers one of which is chosen as correct is very inefficient. Sim-ple changes to the format can treble the information gained. Information theory is the tool for assessing the information content of electronic and other communications. The "bit" is the unit of information and equals on true/false choice. I have applied basic information theory to objective questions. The standard MCQ with a stem, a choice of 4 possible answers (mark one answer true) and an expected correct answer rate of 70% yields 1.36 bits. A MTFQ with 4 choices where the student must answer true/false to every possible answer gives 3.52 bits of information. By adding a "don't know" option the same MTFQ gives 4.72 bits of information, 350% of the standard MCQ. Thirty MTFQ with don't know give the same in-formation about students' knowledge as 100 standard MCQs. The effort needed to set, sit and mark is the same for both. Small changes to the format of objective questions give large gains in efficiency. We should balance these gains against possible disadvantages.
View less >
Journal Title
Creative Education
Volume
4
Issue
6A
Copyright Statement
© 2013 The author and SciRes. This is an Open Access article distributed under the terms of the Creative Commons Attribution 3.0 Unported (CC BY 3.0) License (http://creativecommons.org/licenses/by/3.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Subject
Medical and Health Sciences not elsewhere classified
Education