Efficient Data Pruning via Early-Stage Prediction Stability and Balanced Sampling

Recently, deep learning has achieved remarkable performance gains across a wide range of vision tasks, largely driven by increasingly large-scale training datasets. However, recent studies have shown that many datasets contain redundant or even mislabeled samples, which not only provide limited benefit for model learning but also incur substantial computational costs. Data pruning has therefore emerged as an effective strategy for improving training efficiency by selecting representative subsets of training data. A major limitation of existing data pruning methods is the high computational overhead required to compute sample scores, which contradicts the original goal of reducing overall training cost. To address this issue, we propose an efficient data pruning framework called Early-Stage Prediction Stability, which evaluates sample scores based on prediction stability during the early stages of training. In addition, we introduce a Balanced Sampling Strategy to prevent performance degradation under high pruning ratios. On CIFAR-10, under a 90% pruning ratio, the proposed method reduces the sample scoring time by approximately 66% while achieving a +4.19% accuracy improvement over the previous state-of-the-art method. Extensive experiments on CIFAR-10 and CIFAR-100 demonstrate that the proposed method consistently outperforms existing state-of-the-art pruning approaches across multiple pruning ratios. Furthermore, under 20% label noise on Tiny-ImageNet, the proposed method effectively removes noisy samples and reduces overall training time while achieving state-of-the-art accuracy. The source code is publicly available at https://github.com/Hu210/esps-data-pruning