Invalid voting, meaning blank and spoiled ballots, is a regular phenomenon in democracies around the world. When its share is larger than the margin of victory or greater than the vote share of some of the large parties in the country, invalid voting becomes a problem for democratic legitimacy. This article investigates its determinants in 417 democratic parliamentary elections in 73 countries on five continents from 1970 to 2011. The analysis shows that enforced compulsory voting and ethnic fragmentation are strong predictors for invalid voting while corruption has less impact. Our findings suggest that the societal structure is crucial in understanding invalid voting as a problem for democratic legitimacy because greater social diversity seems to lead to either a greater rate of mistakes or lesser attachments of social groups to the democratic process. Thus, rising levels of invalid voting are not only concerning in themselves but also for the divisive factors driving them.