Nowadays, due to the rapid population expansion, food shortage has become a critical issue. To stabilize food production, preventing crops from being attacked by pests is very important. In general, farmers use pesticides, however, the improper use also kills insects such as bees that are beneficial to crops. If the number of bees is too few, the supplement of food in the world will be short. Besides, excessive pesticides seriously pollute the environment. Accordingly, farmers need a machine that automatically recognizes the pests. Recently, deep learning is popular because of its effectiveness in the field of image classification. In this paper, we propose an efficient model called ExquisiteNet to recognize the pests. ExquisiteNet mainly consists of two blocks. One is double fusion with squeeze-and-excitation-bottleneck block (DFSEB block), and the other is max feature expansion block (ME block). ExquisiteNet only has 0.98 M parameters and its computing speed is fast almost the same as SqueezeNet. To evaluate our model's performance, the model is testedd on a benchmark pest dataset called IP102. The model achieves a higher accuracy of 52.32% on the test set of IP102 without any data augmentation than that of many state-of-the-art models such as ResNet101, ShuffleNetV2, MobileNetV3-large, EfficientNet, and so on.