Skip Navigation

Your Environment. Your Health.

University of Pittsburgh at Pittsburgh

A Unified Sensor System for Ubiquitous Assessment of Diet and Physical Activity

Mingui Sun

Project Description


Obesity is associated with a large variety of genetic and environmental factors. In order to understand the etiology of this condition and develop effective weight management programs, accurate acquisition of diet and physical activity data in the free-living environment is essential. Currently, self-reporting has been the primary method for data acquisition which cannot accurately reflect the habitual behavior of individuals in real life.  As a result, lacking assessment tools that produce unbiased, objective data has significantly hampered the progress of obesity research.


We have proposed a novel application of the multimedia technology to the study of obesity. It is based on electronic chronicle (or e-chronicle) which provides an easily accessible electronic memory of individuals’ daily events. With assistance from video processing algorithms, obesity researchers and clinicians can extract the events related to diet and physical activity, and evaluate these events objectively.


Currently, we have designed and constructed a small wearable computer called an eButton to implement the e-chronicle system. Our device, which contains a miniature camera, a GPS sensor, a 3-axis accelerometer, a 3-axis gyroscope, a light intensity sensor, and several other sensors, is worn below the neck as a regular sized chest pin. It is designed to be almost completely passive to the research participant, and thus will not intrude on or alter the participant’s daily activities.


eButton collects visual data in front of the participant, together with other sensor data, and stores them on a memory card in the device. In the study of diet, for example, the data stored on the card are transferred regularly to the dietitian’s computer where extensive multimedia processing will be performed by software to minimize dietitian’s workload and maximize data processing accuracy.


Possible eating episodes in the video are semi-automatically segmented allowing a visual selection by the dietitian. For each selected episode, food and its portion size are determined computationally assisted by the dietitian. Once the food and portion size are determined, the system uses a food database, such as the United States Department of Agriculture Food and Nutrient Database for Dietary Studies, to determine calories and nutrients. 


The eButton can be used to evaluate not only diet, but also physical activity, sedentary events, and built environment. This wearable computer and its data processing system have provided a powerful new platform to study lifestyle, human behavior, and exposure biology.


See this project's publications and patents 

Back to Top