Atrial fibrillation (AF), being the most prevalent arrhythmia around the world, is a significant health concern considering an aging population and increasing prevalence of its risk factors such as hypertension and obesity. It is estimated that AF increases the risk of stroke by about five times and the risk of its recurrence by two-fold. AF remains undetected in up to 30% of cases due to its asymptomatic and paroxysmal nature, and lack of routine screening. We present a novel AF risk stratification framework using brain magnetic resonance imaging (MRI) to identify the underlying AF in post-stroke patients and assist in preventing secondary ones. By analyzing the infarct patterns of these patients using a multitask learning framework (adopting segmentation and classification losses simultaneously), our proposed model achieves an area under the receiver operating characteristic (AUROC) of 87.48 ± 4.88, demonstrating its capability in discriminating AF patients from others. Since MRI is already an established modality in the stroke treatment and diagnosis framework, this innovative solution incurs no additional costs or tests for the patient. It can effectively select patients at elevated risk for extensive cardiac investigation and definite diagnosis of AF.