Financial hardship may affect up to 30% of cancer survivors, however, little research has addressed the effect of employment change on financial hardship. This study compared the self-reported financial hardship of middle-aged (45–64 years) colorectal cancer survivors (n = 187) at 6 and 12 months following diagnosis with that of a matched general population group (n = 355). Colorectal cancer survivors were recruited through the Queensland Cancer Registry, Australia; data from the Household Income and Labour Dynamics in Australia (HILDA) Survey were used for the general population group. Pearson chi-square tests were used to assess the differences in proportions between the two groups and McNemar tests to assess differences across time among the same group. Generalised linear modelling was performed to produce prevalence ratios. A higher proportion of workers with colorectal cancer reported financial strain (money shortage for living essentials) at 6 months (15%) but eased and was comparable to the comparison group at 12 months (7%). Middle-aged working cancer survivors who ceased or reduced work were more likely to report not being financially comfortable, compared with those who had continued work (adjusted prevalence ratio 1.66, 95%CI: 1.12, 2.44) at 12 months. Health professionals, employers and government services should address the impact of impaired employment on financial hardship among cancer survivors.