Multiple Master-Slave FPGA Architecture of a Stereo Visual Odometry

Estimating relative camera pose is the key problem of visual odometry (VO). To achieve better efficiency, sparse key-points are usually relied on for the estimation. Yet, feature extraction and matching are still computationally demanding, hindering the overall VO from real-time processing. Exploiting the superiorities of an FPGA in terms of high efficiency, low power consumption, and low cost, this paper proposes a multiple master-slave FPGA architecture for an SIFT-based stereo VO. The master-slave design enables high reconfiguration for the data throughputs among various modules. These modules include SIFT, matching, pose estimation, and their corresponding controllers. In the SIFT module, hardware implemented image pyramid is proposed, where scales are determined off-line via a minimization approach. Local linear exhausted search (LES) matching is considered for both the stereo and the frame matching. In the pose estimation module, a novel hardware design of deriving closest orthogonal matrix for 3D-3D correspondences of relative pose estimation is proposed. Experimental results show that 33.2 fps can be achieved using KITTI dataset without the need of a large number of hardware resources. The proposed reconfigurable design also facilitates its expansions of adopting CCD cameras as well as developing SLAM and other applications.