Purpose: This aim of the study is to evaluate the survival function and hazard risks of delayed adjuvant chemotherapy (ChT) to distant recurrence risk in patients with non-metastatic rectal cancer treated with neoadjuvant chemoradiotherapy (CRT) and surgery. Methods: A single tertiary hospital retrospective cohort study of a duration of 5 years between January 2012 and December 2016 was performed. As no previous study shown a temporal relationship of delay to adjuvant/systemic ChT leading to increased risk of metastatic disease, we compared between our proposed cut-off with the median and mean value determined by our dataset. Time to event analysis and log rank tests were conducted. Results: A total of 269 patients with rectal cancer were identified. Two hundred eighteen patients were ineligible, leaving 51 patients for final analysis. Patients in the non-delayed group at 23 (proposed) and 25 (median) weeks’ cut-off reported better 5 years’ disease free survival (DFS) compared with the delayed group by 4.1% and 0.8%. Inversely, at the cut-off 28 (mean) weeks, the delayed group had a better DFS by 4.4%. Females and patients less than 60 years old had better 5-year DFS by 22.8% and 24%. Delayed group has a higher hazard risk ratio (HR) of 1.28 of distant recurrence compared with non-delayed at 23 weeks’ cut-off. Conclusion: This study has demonstrated delaying a patient to adjuvant ChT will lower their DFS and increase their HR compared with those whose treatment is not delayed. We have long been too focused on local control; hence, priority needs to be shifted to efforts in managing potential distant disease in a timely manner.