High speed trains are promoted all over the world as an alternative to aircraft (or conventional trains) for travel distances from a few hundred to a (few) thousand kilometers. Long travel time broadens the scale of activities potentially affected by train noise and sounds from other passengers. In this experiment, the effect on task performance caused by the combination of train noises and passengers' chatting was investigated. While being exposed to different noise combinations, Chinese as well as Dutch speaking participants were asked to perform a series of calculation and memory tasks as quickly and as accurately as possible. Results mainly show that (i) response time and accuracy are independent variables for both tasks, with the former showing the largest effect caused by background noise; (ii) a model based on saliency and attention can predict a significant part of the influence of combined noise (train and speech) on task performance; and (iii) in retrospect, the majority of participants report the speech sound to have been either equally or more annoying although the addition of speech changed the overall L Aeq by less than 0.1 dB(A).