Do instructional manipulation checks measure inattention or miscomprehension?
File version
Accepted Manuscript (AM)
Author(s)
Paas, L
Dolnicar, S
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
Size
File type(s)
Location
Abstract
Instructional Manipulation Checks (IMCs) are intended to detect inattention, a common occurrence in survey responding. We use eye tracking to empirically assess the attention that survey respondents dedicate to a short and a long IMC. We find that all 21 respondents pass the short IMC. In contrast, six respondents fail the long IMC. Our eye-tracking results show that some respondents who fail the long IMC check allocate more cognitive effort and time to processing this check than others, suggesting that lack of comprehension rather than inattention may be the relevant underlying mechanism for their failure. For other respondents failing the IMC, lack of attention and speeding is more likely to be the culprit. These findings culminate into the suggestion that future researchers should assess the time that respondents failing an IMC dedicated to this check to distinguish between miscomprehension (long time) and inattention (short time).
Journal Title
International Journal of Social Research Methodology
Conference Title
Book Title
Edition
Volume
Issue
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
© 2022. This is an Accepted Manuscript version of the following article, accepted for publication in International Journal of Social Research Methodology. Nazila Babakhani, Leo Paas & Sara Dolnicar (2022) Do instructional manipulation checks measure inattention or miscomprehension?, International Journal of Social Research Methodology, DOI: 10.1080/13645579.2022.2111064. It is deposited under the terms of the Creative Commons Attribution-NonCommercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Item Access Status
Note
This publication has been entered in Griffith Research Online as an advanced online version.
Access the data
Related item(s)
Subject
Gender studies
Sociology
Persistent link to this record
Citation
Babakhani, N; Paas, L; Dolnicar, S, Do instructional manipulation checks measure inattention or miscomprehension?, International Journal of Social Research Methodology, 2022