Mobile Food Intake Visualization and Voice Recognize (FIVR)
Inadequate dietary intake assessment tools hamper studying relationships between diet and disease. Methods suitable for use in large and small epidemiologic studies (e.g., dietary recall, food diaries, and food frequency questionnaires) are subject to considerable inaccuracy, and more accurate methods (e.g., metabolic ward techniques, doubly-labeled water) are prohibitively costly and/or labor-intensive for use in population-based studies. A simple, inexpensive and convenient, yet valid, dietary measurement tool is needed to provide more accurate determination of dietary intake in populations.
This project is developing and testing a new assessment tool called FIVR (Food Intake Visualization and Voice Recognizer) that uses a novel combination of innovative technologies: advanced speech recognition and visualization techniques in an electronic system to automatically record and evaluate food intake. FIVR uses cell phones to capture both voice recordings and photographs of dietary intake in real-time. These dual sources of data are sent to a database server for recognition processing for real-time food recognition and portion size measurement through speech recognition and image analysis.
The FIVR objectives are to fuse existing speech and image recognition techniques into a system that will recognize foods by food type and unique characteristics and determine volume by a set of photos showing an image from three angles. Each item identified will be matched to an appropriate food item within a food composition database and by automatically determining food volume nutrient intake will be computed. Researchers will view the resulting analysis through a web-based dietary analysis program.
The proposed protocol incorporates three discrete phases:
- Technology development, integration, and testing;
- Validity testing with a controlled diet (metabolic ward study); and
- Real-world utility testing.
Validity of the data collected will be judged by how closely the nutrient calculations match the known composition of the metabolic ward diets consumed and usability judged by real-world testing of the system.Currently FIVR can calculate volume on almost 200 foods with more foods being added on an ongoing basis. To quantify volume estimation accuracy, we performed a test on 14 foods in 6 food categories (breakfast foods, snack foods, lunch foods, meats, fruits, and vegetables). The test involved 101 sets of food images (each set containing 5 images), captured under different lighting variations with results compared against a ground truth measure of the food volume using water displacement. We compared our original FIVR1 system against our current improved FIVR2 system using both manual segmentation and automatic techniques. Our results were:
- Manual Comparison FIVR1: 12.66% to FIVR2: 10.46% average error
- Automatic Comparison FIVR1: 33.89% to FIVR2: 15.50% average error
After completing the current project, including the free-living validation study, Viocare plans to commercialize FIVR as a product available to researchers, clinicians, and as part of corporate wellness programs. A number of researchers have plans to use FIVR within their upcoming cohort studies. FIVR will also become a component of Viocare’s VioWell architecture, supporting clinical programs to improve an individual’s dietary behavior. FIVR has the potential to establish a method of highly accurate dietary intake assessment suitable for cost-effective use at the population level, and thereby advance crucial public health objectives.