Electroencephalogram (EEG), boasting the advantages of portability, low cost, and hightemporal resolution, is a non-invasive brain-imaging modality that can be used to measure different brain states. However, EEG recordings are always contaminated with artifacts from different sources other than neurons, which renders EEG data analysis more difficult, and which potentially results in misleading findings. Therefore, it is essential for many medical and practical applications to remove these artifacts in the preprocessing stage before analyzing EEG data. In the last thirty years, various methods have been developed to remove different types of artifacts from contaminated EEG data; still though, there is no standard method that can be used optimally, and therefore, the research remains attractive as well as challenging. This paper presents an extensive overview of the existing methods for ocular, muscle, and cardiac artifact identification and removal with their comparative advantages and limitations. We also reviewed the schemes developed for validating the performances of algorithms with simulated and real EEG data. In future studies, researchers should focus not only on the combining of different methods with multiple processing stages for efficient removal of artifactual interferences but also on the development of standard criteria for validation of recorded EEG signals.