This paper investigates the history of sexual violence in Australian male prisons. It traces a genealogy of sex between incarcerated men from the penal colonies to the 1990s, drawing on a range of criminal justice and sexological and cultural records to show how sexual violence inflected both conceptions and experiences of homosexual prisoners. We outline how these inmates shifted from being perceived as an institutional threat to a victim of the institution, an uneven process propelled by competing and contradictory ideas about the constitution and practice of sexual behaviours, identities and power behind bars.