Detection of bus route number is a very important issue for assisting visually impaired people to take the bus. This paper proposes an intelligent approach to detect bus route numbers to help visually impaired people to "see" the bus route number of a moving bus. Current e-bus stations in Taipei, Taiwan, are not very robust in bad weather. For visually impaired people, it is very difficult know which bus route number is approaching the bus station. Some past research used RFID, GIS, sound, and image-based methods to help visually impaired people to "see" bus route number. However, the development of bus route number reading devices for visually impaired people in real time is still at an early stage. In this paper, we propose an efficient and effective learning-based method to detect the bus route number in displayed on the bus façade panel of the moving bus. Experimental results show that the proposed method can reduce time complexity and achieve high bus route number detection rates.