This paper presents a probabilistic graphical model to formulate and deal with video-based face recognition. Our formulation divides the problem into two parts: one for likelihood measure and the other for transition measure. The likelihood measure can be regarded as a traditional task of face recognition within a single image, i.e., to recognize who the current observing face image is. In our work, two-dimensional linear discriminant analysis (2DLDA) is employed to judge the likelihood measure. Moreover, the transition measure estimates the probability of the change from a false recognition at the previous stage to the correct person at the current stage. Our approach for transition measure does not only consider the visual difference among persons according to the training face images but also involve prior information of the pose change in video frames. We also provide several experiments to show the efficiency of our proposed approach in this paper.