Circle detection is one of the most important problems in image processing since circular objects, such as balls, are observed frequently in many natural and artificial environments and problems such as light variations, occlusions, shadows, circle-shaped objects, and real-time processing have to be managed. The previous methods are either edge-based which generally suffer from processing burden or color-based and cannot deal with color variations suitably. This paper presents a real-time ball detection framework that uses both color information and shape information together to detect and track a ball robustly. The results demonstrate superiority of the proposed method over the previous. After classifying the color space, image segmentation, building regions, and extracting objects, to reduce the computation, it is focused on finding the green horizon to eliminate a part of the image, which is beyond the field border. Afterward some filters based on circle-fitting methods, image moments are applied to detect and track the ball. The results show that these filters are real-time, robust against occlusion and are able to track the ball even for distances more than six meters.