An analysis of facial movement tracking in ordinary human-computer interaction

Automatic tracking of facial movement is potentially important as a non-invasive source of physiological data in Affective Computing applications. Facial movement tracking software is becoming commercially available and affordable. This paper explores the association between facial and physiological responses to computer-based events, and the viability of facial movement tracking in detecting and distinguishing qualitative differences in users' facial movements under normal conditions of computer use.

Fifteen participants took a web-based quiz. The quiz contained two relatively ordinary HCI events as stimuli: an alert intended to evoke surprise, and questions with high affective content intended to evoke amusement. From previous findings, the alert was expected to be the stronger of the two stimuli. Participants' physiological arousal was recorded and their faces videoed. The videos for the periods around each event were analysed by commercially available facial movement tracking software.

Human judges considered participants' faces to have responded to both stimuli, but more to the stronger of the two stimuli. Facial response did not always concur with physiological arousal. The tracker detected reactions to the stronger stimulus but had mixed success with the weaker stimulus. The tracker also generated different data profiles for two different facial expressions. These findings support the supposition that users' facial expressions can and do respond to ordinary computer-based events, and indicate that facial movement tracking is becoming a viable technique, and is available to non-computer vision specialists