Neuromyelitis optica spectrum disorder (NMOSD) and multiple sclerosis (MS) are inflammatory diseases of the CNS. Overlap in the clinical and MRI features of NMOSD and MS means that distinguishing these conditions can be difficult. With the aim of evaluating the diagnostic utility of MRI features in distinguishing NMOSD from MS, we have conducted a cross-sectional analysis of imaging data and developed predictive models to distinguish the two conditions. NMOSD and MS MRI lesions were identified and defined through a literature search. Aquaporin-4 (AQP4) antibody positive NMOSD cases and age- and sex-matched MS cases were collected. MRI of orbits, brain and spine were reported by at least two blinded reviewers. MRI brain or spine was available for 166/168 (99%) of cases. Longitudinally extensive (OR = 203), “bright spotty” (OR = 93.8), whole (axial; OR = 57.8) or gadolinium (Gd) enhancing (OR = 28.6) spinal cord lesions, bilateral (OR = 31.3) or Gd-enhancing (OR = 15.4) optic nerve lesions, and nucleus tractus solitarius (OR = 19.2), periaqueductal (OR = 16.8) or hypothalamic (OR = 7.2) brain lesions were associated with NMOSD. Ovoid (OR = 0.029), Dawson’s fingers (OR = 0.031), pyramidal corpus callosum (OR = 0.058), periventricular (OR = 0.136), temporal lobe (OR = 0.137) and T1 black holes (OR = 0.154) brain lesions were associated with MS. A score-based algorithm and a decision tree determined by machine learning accurately predicted more than 85% of both diagnoses using first available imaging alone. We have confirmed NMOSD and MS specific MRI features and combined these in predictive models that can accurately identify more than 85% of cases as either AQP4 seropositive NMOSD or MS.