The aim of this study was to investigate whether variations in the way that pain intensity is measured in patients with neck pain influences the magnitude of pain ratings. The study uses data from 3 longitudinal studies (n = 361 at baseline) on people with neck pain due to whiplash injuries. Pain measures included verbal rating scales, numerical rating scales and a visual analog scale. Different measures asked patient to rate current pain, average pain over 24 hours, over 1 week, or over 4 weeks. Scores were converted to a 0-100 scale and tracked over time, correlations between measures were calculated. Mixed models regression was used to explore the factors which influenced the differences between scores on the measures. Scores on the different measures were significantly different from each other in each dataset (P < 0.02). The effect of recall period was significant in all datasets and the effect of number of response options was significant in 2 of 3 datasets. Pain intensity ratings appear to be sensitive to method of measurement. It is likely the length of recall time (eg, pain today vs. average pain over 4 weeks) has a significant influence on pain ratings. The influence of number of response options is less certain. Systematic reviewers should not uncritically rescale and pool absolute pain scores from instruments with varying scale descriptors or recall periods.