Automatic inspection techniques have been widely employed to achieve high productivity while ensuring high-quality products in steel making industry. In this paper, a vision-based detection framework for automatically detecting different types of steel billet surface defects is proposed. The defects considered in this study includes cratches, corner cracks, sponge cracks, slivers, and roll marks. In the proposed framework, to improve the quality of image acquisition for billet surface, two preprocessing techniques, i.e., automatic identification of ROI (region of interest) and HDR (high dynamic range)-based image enhancement techniques, are proposed. Then, DWT (discrete wavelet transform)-based image feature is extracted from the image to be detected and fused with the other two extracted local features based on variance and illumination to identify each defect on the billet surface. Experimental results have verified the feasibility of the proposed method.