Shadow detection and removal is an important step employed after foreground detection, in order to improve the segmentation of objects for tracking. Methods reported in the literature typically have a significant trade-off between the shadow detection rate (classifying true shadow areas as shadows) and the shadow discrimination rate (discrimination between shadows and foreground). We propose a method that is able to achieve good performance in both cases, leading to improved tracking in surveillance scenarios. Chromacity information is first used to create a mask of candidate shadow pixels, followed by employing gradient information to remove foreground pixels that were incorrectly included in the mask. Experiments on the CAVIAR dataset indicate that the proposed method leads to considerable improvements in multiple object tracking precision and accuracy.