Background: Lack of evidence about the external validity of discrete choice experiments (DCEs) is one of the barriers that inhibit greater use of DCEs in healthcare decision making. Objectives: To determine whether the number of alternatives in a DCE choice task should reflect the actual decision context, and how complex the choice model needs to be to be able to predict real-world healthcare choices. Methods: Six DCEs were used, which varied in (1) medical condition (involving choices for influenza vaccination or colorectal cancer screening) and (2) the number of alternatives per choice task. For each medical condition, 1200 respondents were randomized to one of the DCE formats. The data were analyzed in a systematic way using random-utility-maximization choice processes. Results: Irrespective of the number of alternatives per choice task, the choice for influenza vaccination and colorectal cancer screening was correctly predicted by DCE at an aggregate level, if scale and preference heterogeneity were taken into account. At an individual level, 3 alternatives per choice task and the use of a heteroskedastic error component model plus observed preference heterogeneity seemed to be most promising (correctly predicting >93% of choices). Conclusions: Our study shows that DCEs are able to predict choices—mimicking real-world decisions—if at least scale and preference heterogeneity are taken into account. Patient characteristics (eg, numeracy, decision-making style, and general attitude for and experience with the health intervention) seem to play a crucial role. Further research is needed to determine whether this result remains in other contexts.