Multimodal Segmentation of Lifelog Data

A personal lifelog of visual and audio information can be very helpful as a human memory augmentation tool. The SenseCam, a passive wearable camera, used in conjunction with an iRiver MP3 audio recorder, will capture over 20,000 images and 100 hours of audio per week. If used constantly, very soon this would build up to a substantial collection of personal data. To gain real value from this collection it is important to automatically segment the data into meaningful units or activities. This paper investigates the optimal combination of data sources to segment personal data into such activities. 5 data sources were logged and processed to segment a collection of personal data, namely: image processing on captured SenseCam images; audio processing on captured iRiver audio data; and processing of the temperature, white light level, and accelerometer sensors onboard the SenseCam device. The results indicate that a combination of the image, light and accelerometer sensor data segments our...

• Type: Conference
• Year: 2007
• Where: RIAO
• Authors: Aiden R. Doherty, Alan F. Smeaton, Keansub Lee, Daniel P. W. Ellis
• Keywords: Accelerometer Sensor, Data Sources, Information Technology, Personal Data, RIAO 2007