Real-time face tracking based on facial feature matching

In this paper, we propose a real-time system to extract and track people's facial features effectively. It can also resist rotation, scaling, and parallax of the image. When camera captures video frame, the proposed system can recognize where the face is, and then uses our Dynamic Radial Kernel to record and match facial features in each frame. After getting all the facial features from those frames, we can realize what the user's movement is happening, such as face direction changing, face rotation, and depth changing, because every frame is in the same time sequence. At last, we map the 2D coordinate to 3D space by perspective transform. The experimental result shows that the proposed method is successful. It can recognize human facial features in several environments robustly. In addition, we implement a human interface system using the proposed method to display an augmented reality (AR) application. In the future work, we will try to improve our algorithm of feature recording, matching, and make it suitable to any content of image.