A novel approach of Model-based Building Reconstruction (MBBR) from topographic maps and LiDAR data called Floating Models is proposed in this paper. Floating models are a series of pre-defined primitive models which are floating in the space. Its size is adjustable by shape parameters, while its location and rotation is controlled by pose parameters. A building is reconstructed by adjusting these model parameters so the wire-frame model adequately fits into the building's outlines among the topographic maps, LiDAR data, aerial photos and DEM. This model-based reconstruction provides good constraints to the shape of the model in contrary to the data-based approach. In this paper, the model parameters are rearranged into two groups: plane and height parameters. The plane parameters are determined by fitting the top or bottom boundary of the model to the topographic maps. The height parameters are decided by fitting the top surface of the model to the lidar data and interpolating the datum point's height from DEM. The proposed reconstructing procedure is semi-automated. First, the operator chooses an appropriate model and approximately fit to the building's outlines on the topographic map. Second, the computer computes the optimal fit between the model and the topographic map based on an ad hoc least-squares model fitting algorithm. Third, the computer computes the roof or ridge height form the lidar points within the roof's boundary. Finally, the model parameters and standard deviations are provided, and the wire-frame model is superimposed on all overlapped aerial photos for the operator to check the result. The operator can make any necessary modification by adjusting the corresponding model parameter. We select a small urban area of Taipei City for testing the proposed approach. The fitting result is compared to the traditionally photogrammetric result. Most of the modern buildings can be modeled smoothly, and fitting result achieves the photogrammetric accuracy.