The relationship between farm size and dietary diversity has recently gained attention among development practitioners and policymakers. To study this issue, we utilized cross-sectional data from 900 farm households in Northern Ghana. The analysis employs three econometric approaches—ordinary least squares, two-stage least squares, and instrumental variable (IV) quantile regression. We subjected our estimates to Conley-Hansen-Rossi bound tests to assess the validity of total farmland under household control as an IV for farm size, which serves to illustrate the robustness of the causal inferences linking farm size to household dietary diversity. The results show that a 1-ha increase in cultivated land improves household dietary diversity score by 3.221 units. The effect of cultivated land on dietary diversity is enhanced by market information, use of improved seeds, chemical fertilizers and herbicides, and overall household asset value. In conclusion, the results indicate that farm size increases household dietary diversity when moving along the dietary diversity distribution. This highlights the importance of considering the influence of farm size at different points along the conditional dietary diversity score distribution, as relying solely on mean effects may obscure important counteracting effects.