Anomaly detection has become an important topic in Hyperspectral Imagery (HSI) analysis in the last two decades with the advantage of detecting the targets surrounding in diverse backgrounds without prior knowledge. HSIs usually have complex and redundant spectral signals due to the complicated land-cover distribution. Generally, it is difficult to estimate the background accurately, and distinguish the anomaly targets. The performances of traditional algorithms are difficult to meet the requirements. In this paper, we propose a novel anomalous component extraction framework for hyperspectral anomaly detection based on Independent Component Analysis (ICA) and Orthogonal Subspace Projection (OSP). In the proposed method, the brightest anomalous component is extracted to initialize the projection vector, by which the performance of ICA can be improved greatly. Moreover, the Independent Component (IC) containing the most abnormal information can be obtained according to the vector. Besides, The OSP algorithm is applied to suppress the background components in the remaining data. Then the data are iteratively processed by ICA to extract the anomalous component subtly. Therefore, in the initialization process, the possible situation of detecting the pixels in the same position can be effectively avoided, and the interference of the last iteration procedure can be cut down greatly, helping to optimize the detection. Finally, the experimental results show that the proposed framework achieves a superior performance compared to some of the state-of-the-art methods in the field of anomaly detection.