Food Calorie and Nutrition Analysis System based on Mask R-CNN

Over the past few decades, obesity has become a serious problem. Obesity is associated with many of the leading causes of death, such as chronic diseases including diabetes, heart disease, stroke, and cancer. The most effective way to prevent obesity is through food intake control, which involves understanding food ingestion, including the nutrients and calories of each meal. To assist with this issue, this study develops a food calorie and nutrition system that can analyze the composition of a food based on a provided image. Further, we introduce a newly collected dataset, Ville Cafe, for food recognition. This dataset contains 16 categories with 35,842 images, including salad, fruit, toast, egg, sausage, chicken cutlet, bacon, French toast, omelet, hash browns, pancake, ham, patty, sandwich, French fries, and hamburger. The system is based on a Mask Region-based Convolutional Neural Network (R-CNN) with a union postprocessing, which modifies the extracted bounding boxes and masks, without the non-maximum suppression (NMS), to provide a better result in both analytics and visualization. The recognition accuracy for the combination of Ville Cafe and the Food-256 Datasets was 99.86%, and the intersection over union (IoU) was 97.17%. The food weight estimation experiment included eight classes: salad, fruit, toast, sausage, bacon, ham, patty, and French fries. These classes respectively had 40, 40, 44, 40, 41, 49, 26, and 40 data points, adding up to 320 data points for the linear regression model. In the experimental results, the average absolute error was 8.22, and the average relative error was 0.13.