An iOS App for Automatic Calorie Estimation.
A food photo generally includes several kinds of food dishes. In order to recognize food images including multiple dishes, we need to detect each dish in food images. Meanwhile, in recent years, the accuracy of object detection has improved drastically by the appearance of Convolutional Neural Network (CNN). In this demo, we present an automatic calorie estimation app, DeepCalorieCam, running on iOS. DeepCalorieCam can estimate the calorie after detecting dishes from the video stream captured from the built-in camera of an iPhone. We use YOLOv2 [2] which is the state-of-the-art object detection system using CNN, as a dish detector to detect each dish in a food image, and the food calorie of each detected dish are estimated by image-based food calorie estimation [1].
We implemented the network proposed as a real-time object detection network called YOLOv2 by Redmon et al. [1] and the multi-task CNN proposed as a simultaneous estimation of food categories and calories calorie by Ege et al. [2].
First, we trained the dish detector by fine-tuning YOLOv2[1] with UECFOOD-100. Next, we trained food calorie estimation CNN by fine-tuning ResNet50 with food calorie estimation with 15 categories dataset[2]. After that, we converted the trained models for CoreML and deployed it to iOS.
[1]J. Redmon and A. Farhadi. YOLO9000: Better, Faster, Stronger. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[2]T. Ege and K. Yanai. Simultaneous Estimation of Food Categories and Calories with Multi-task CNN. In Proc. of ACPR International Conference on Machine Vision Applications (MVA), 2017.
