In this paper, we present a method of recovering a high dynamic range (HDR) image from several ones of low range and then calibrating its color. Our goal is not only composing the HDR image with more details clearly, but also reducing the color distortion during the tone mapping process. The algorithm we designed composes multiple color images with different exposure levels of the same scene into a single high dynamic range one by a proposed matrix, named color calibration matrix (CCM). The GretagMacbeth color checker was used to evaluate the color distortions of images captured by a camera under various exposure levels. These camera distortion characteristics were modeled and estimated by the CCM. The experimental results illustrated that the method we proposed reduces significantly the unnatural color distortion caused by the lightness changes of the captured original images of low dynamic range (LDR). It seemed that the designed algorithm can work well as an accurate HDR recovering method especially under the demand for the correct color appearance.