A Study in Efficiency and Modality Usage in Multimodal Form Filling Systems

The usage patterns of speech and visual input modes are investigated as a function of relative input mode efficiency for both desktop and personal digital assistant (PDA) working environments. For this purpose the formfilling part of a multimodal dialogue system is implemented and evaluated; three multimodal modes of interaction are implemented: "Click-to-Talk", "Open-Mike" and "Modality-Selection". "Modality-Selection" implements an adaptive interface where the system selects the most efficient input mode at each turn, effectively alternating between a "Click-to-Talk" and "Open-Mike" interaction style as proposed in [1]. The multimodal systems are evaluated and compared with the unimodal systems. Objective and subjective measures used include task completion, task duration, turn duration and overall user satisfaction. Turn duration is broken down into interaction time and inactivity time to better measure the efficiency of e...

• Type: Journal
• Year: 2008
• Where: TASLP
• Authors: Manolis Perakakis, Alexandros Potamianos
• Keywords: Efficient Input Mode, Input Mode, Multimodal Dialogue, TASLP 2008