This paper presents a fast and novel algorithm for bilateral symmetry detection based on inverted gradient hash maps (IGHMs). A hash map is an associative array that stores image gradient magnitudes and orientations in the form of an inverted index. This mapping of image gradients to their locations permits points of interest to be located very rapidly without needing to search through the image. Unlike many symmetry operators it is able to detect large-scale symmetry. The method is described and experimentally evaluated against existing methods for bilateral symmetry detection.