In the area of auricular medicine research, researchers believe that human diseases are reflected in corresponding locations on the external ears, known as the positive reactions of auricular points. If some auricular points on a person's external ear exhibit abnormal positive reactions, then the person is being attacked by a corresponding disease. This paper proposes a vision-based auricular diagnosis assistance system to help doctors detect diseases by discovering the visual positive reactions of auricular points on patients' external ears. The proposed vision-based auricular diagnosis assistance system includes visual positive reaction detection and disease identification. First, external ear images are input into the system to detect the visual positive reactions of the auricular points on the external ear using an improved version of the U-Net model. Improvements of the U-Net model include batch standardization, atrous convolution, convolution stage reduction, and multi-expansion-rate integration. Second, the detection results of visually positive reactions are then used to identify the diseases. This study identified nine types of diseases through their corresponding positive reactions, including hepatitis, mastitis, cervicitis, prostatitis, frontal headache, migraine, occipital headache, vertex headache, and headache. The dataset used in this study was collected by the authors and called the CVIU 108 EAR Dataset. The experimental results show that the disease diagnosis accuracy rate of the proposed system is 99.60%.