Previous research shows inconsistency in clinician-assigned diagnoses of Autism Spectrum Disorder (ASD). We conducted an exploratory study that examined the concordance of diagnoses between a multidisciplinary assessment team and a range of independent clinicians throughout Australia. Nine video-taped Autism Diagnostic Observation Schedule (ADOS) assessments were collected from two Australian sites. Twenty-seven Australian health professionals each observed two video-recordings and rated the degree to which the individual met the DSM-5 criteria for ASD. There was 100% agreement on the diagnostic classification for only 3 of the 9 video clips (33%), with the remaining 6 clips (66%) reaching poor reliability. In addition, only 24% of the participating clinicians achieved ‘good’ or ‘excellent’ levels of agreement (Cohen’s kappa > 0.6) with the original ASD assessment. These findings have implications for clinical guidelines for ASD assessments.