—This study investigates a method for estimating a driver’s spontaneous frustration in the real world. In line with a specific definition of emotion, the proposed method integrates information about the environment, the driver’s emotional state, and the driver’s responses in a single model. Driving data are recorded using an instrumented vehicle on which multiple sensors are mounted. While driving, drivers also interact with an automatic speech recognition (ASR) system to retrieve and play music. Using a Bayesian network, we combine knowledge on the driving environment, assessed through data annotation, speech recognition errors, driver’s emotional state (frustration), and driver’s responses measured through facial expressions, physiological condition, and gas- and brake-pedal actuation. Experiments are performed with data from 20 drivers. We discuss the relevance of the proposed model and features of frustration estimation. When all of the available information is used, ...