As global efforts to address COVID-19 become more urgent, the public discourse on attitudes towards immigrants has also become more polarized. In such a heightened social and political context, we study the impact of intergroup contact in political jurisdictions with different shares of minority populations. Drawing on data from the Australian Election Study and the Census, we find that less intergroup contact with neighbours from minority birthplace, language and citizenship backgrounds is correlated with higher anti-immigrant sentiments. These results have significant implications for immigrant-receiving countries with existing legislation and policies that direct immigrants to move to areas of lower population density, lower presence of immigrants, and less previous exposure to international migration. The evidence presented in this study suggests that without sufficient political and policy sensitization to the issues for immigrants living in these areas, there could be detrimental social and political impacts to both immigrants and society more generally.