The Gestural Control of Audio Processing

Gesture enabled devices have become so ubiquitous in recent years that commands such as ‘pinch to zoom-in on an image’ are part of most people’s gestural vocabulary. Despite this, gestural interfaces have been used sparingly within the audio industry. The aim of this research project is to evaluate the effectiveness of a gestural interface for the control of audio processing. In particular, the ability of a gestural system to streamline workflow and rationalise the number of control parameters, thus reducing the complexity of Human Computer Interaction (HCI). A literature review of gestural technology explores the ways in which it can improve HCI, before focussing on areas of implementation in audio systems. Case studies of previous research projects were conducted to evaluate the benefits and pitfalls of gestural control over audio. The findings from these studies concluded that the scope of this project should be limited to two-dimensional gestural control. An elicitation of gestural preferences was performed to identify expert-user’s gestural associations. This data was used to compile a taxonomy of gestures and their most widely-intuitive parameter mappings. A novel interface was then produced using a popular tablet-computer. This facilitated the control of equalisation, compression and gating. Objective testing determined the performance of the gestural interface in comparison to traditional WIMP (Windows, Icons, Menus, Pointer) techniques, thus producing a benchmark for the system under test. Further testing is carried out to observe the effects of graphic user interfaces (GUIs) in a gestural system, in particular the suitability of skeuomorphic (knobs and faders) designs in modern DAWs (Digital Audio Workstations). A novel visualisation method, deemed more suitable for gestural interaction, is proposed and tested.
Semantic descriptors are explored as a means of further improving the speed and usability of gestural interfaces, through the simultaneous control of multiple parameters. This rationalisation of control moves towards the implementation of gestural shortcuts and ‘continuous pre-sets’.