PhD in Computer Science and Creative Technology

End-User Action-Sound Mapping Design for Mid-Air Music Performance

How to design the relationship between a performer's actions an instrument's sound response has been a consistent theme in Digital Musical Instrument (DMI) research. Previously, mapping was seen purely as an activity for DMI creators, but more recent work has exposed mapping design to the musician, with many in the field introducing software to facilitate end-user mapping, democratising this aspect of the DMI design process. This end-user mapping process provides musicians with a novel avenue for creative expression, and offers a unique opportunity to examine how practicing musicians approach mapping design.


Most DMIs suffer from a lack of practitioners beyond their initial designer, and there are even fewer examples that are used by professional musicians over extended periods. The Mi.Mu Gloves are one of the few examples of DMI that are used by a dedicated group of practising musicians, many of whom use the gloves in their professional practice, and a significant aspect of creative practice with the gloves is end-user mapping design. The research presented in this dissertation investigates end-user mapping practice with the Mi.Mu Gloves, and what influences glove musicians' design decisions based on the context of their music performance practice.


Empirical research found marked differences between the mapping design practice of expert and novice glove musicians. Expert musicians design perceptually simple mappings while embellishing them with performative ancillary gesture, and design mappings that minimise performance and system-related errors, with issues around accidental triggering being a major impediment to their expressive mapping design. Meanwhile, novice musicians design mappings with the audience's perception of their mappings in mind, designing mappings that adhere to established conceptual metaphors relating to movement and musical properties, while also finding that system-related errors from an end-user machine learning tool were a major issue. The issue of system-related errors was then investigated, which found that small amounts of system-error caused significant disruption to a musician's ability to acquire skill.


Learning from these findings, a series of design heuristics are presented, applicable for use in the fields of DMI design, mid-air interaction design and end-user mapping design.

Read the full thesis here