Background modeling is the core of event detection in surveillance systems. The traditional Gaussian mixture model has some defects when encountering some situations like shadow interferences, lighting changes, and other problems causing foreground image broken. All of these cases will result in deficiencies of event detection. In this paper, we propose a new background modeling method to solve these problems. The model features of our method are the combination of texture and color characteristics, hysteresis thresholding, and the motion estimation to recover broken foreground objects.