To achieve the goal of Industry 4.0 for steel industry, intelligent automated manufacturing becomes increasingly important. One of the key steps on the automatic production line of steel products is automatic inspection of surface detects. Recent trends in vision-based surface inspections of steel products have shown that the use of 3D information would be more beneficial than only utilizing two dimensional surface data. In this paper, a 3D modeling framework is proposed to capture 3D information of steel billet images. Our major goal is to obtain the depth information for the billet images to be inspected for possibly further applications (e.g., inspection of surface defects). The presented results have shown the feasibility of the proposed framework.