Many classification tasks benefit from integrating manifold learning and semi-supervised learning. By formulating the learning task in a semi-supervised manner, we propose a novel objective function that integrates the manifold consistency of whole dataset and the hinge loss of class label prediction. This formulation results in a SVM-alike task operating on the kernel derived from the graph Laplacian, which can capture the intrinsic manifold structure of the whole dataset and maximize the margin separating the labelled examples. Results on synthetic tasks, face and handwritten digit recognition all show significant performance gain. The performance gain is particularly impressive when only a small training set is available, which is often the true scenario of many real-world pattern recognition problems.