Traditional Support Vector Regression (SVR) Machine acts as approximating a regression function. This paper, however, proposes a novel multi-class classification approach based on the SVR framework, called Support Vector Regression Machine with Consistency (SVRC). The contributions of this paper are: (1) To implement multi-class classification task, were place the margin term with its l1 norm in the SVR framework; (2)To make the training data within the same class possess approximate contributions for the test sample reconstruction and thus improve the robustness, we construct a consistent matrix employing the class information and introduce the penalty term using it; (3) To pay more attention to using fewer possible classes to represent the test sample, and thus improve the accuracy of the test sample reconstruction, we utilize the corresponding local neighborhood relationship of the test sample to design a selection matrix. Experimental results demonstrate that the performance of the proposed method is much better than that of some existing multi-class classification approaches.