In this paper, we integrate shadow information into the background model of a scene in an attempt to detect both shadows and foreground objects at a time. Since shadows accompanying foreground objects are viewed as parts of the foreground objects, shadows will be extracted as well during foreground object detection. Shadows can distort object shapes and may connect multiple objects into one object. On the other hand, shadows tell the directions of light sources. In other words, shadows can be advantageous as well as disadvantageous. To begin, we use an adaptive Gaussian mixture model to describe the background of a scene. Based on this preliminary background model, we extract foreground objects and their accompanying shadows. Shadows are next separated from foreground objects through a series of intensity and color analyses. The characteristics of shadows are finally determined with the principal component analysis method and are embedded as an additional Gaussian in the background model. Experimental results demonstrated the feasibility of the proposed background model.