A natural experiment is reported in which the performance of two cohorts of undergraduate students on identical tests of cognitive achievement is compared under differing conditions of proctoring and modes of administration, total N = 302. Between-subjects and within-subjects analyses reveal clear effects of test score inflation in the unproctored conditions compared with formal, proctored, examination conditions. These results caution against the use of unproctored cognitive testing in high-stakes situations, including on-line assessment.