Muslims are now arguably the most widely debated and feared segment of the Australian community but they are also its most long-standing non-indigenous segment. In Australia we are able to draw on a long and primarily positive contact history between Muslim and non-Muslim Australians that makes nonsense of the paranoid nationalism with which the Howard government wanted to protect a way of life from 'recent invaders'. There are deep histories underlying some of the highly debated 'border control' phenomena such the Tampa refugees and 'Timorese poachers'. The way we understand our histories also shapes the way in which we can imagine our futures and the fantasy of a white Australian history does not stand up to historical investigation. Keywords: Muslims in Australia; Aboriginal history; Asian-Aboriginal history; border control