In this paper, we investigate the problem of automatically constructing characters' social network from movies. Unlike existing approaches that use co-appearance information to measure the relationship between two characters, we argue that a method that describes the characters' interaction, rather than the co-appearance, makes more sense. We propose a new scheme that quantifies the interaction of characters by the use of film-editing cues, based on which we construct the characters' social network. Experiments on real-world data validate the effectiveness of the proposed method. In addition, we show an application of discovering characters' social clusters enabled by the automatically constructed social network.