The development of mobile platform has raised an emergent requirement for face-related multimedia applications. However, as the basis of such applications, face detection and tracking still suffers from large facial pose variation and limited computation resource. In this paper, a multi-view and real-time face tracking system is presented for mobile platform. First, three basic detectors are trained by local binary pattern (LBP) and Boosting algorithm. These detectors are then flexibly expanded for multi-view face detection through the rotation facility of LBP. To further accelerate the face tracking process, a robust facial pose estimation algorithm and the face matrix partition scheme is proposed. The experimental results on handset show the superior performance of the proposed method.