Interfaces for musical expression based on simulated physical models
- Publication Type:
- Thesis
- Issue Date:
- 2009
Open Access
Copyright Clearance Process
- Recently Added
- In Progress
- Open Access
This item is open access.
This thesis is concerned with the design of interactive virtual musical instruments
intended to augment acoustic instruments in live performance. For the purposes of
this work, a virtual musical instrument is defined as a computer system designed to
facilitate musical expression and/or exploration. The aim of the research is to develop
understanding of the nature of virtual instruments and how musicians interact with
them. The approach has been to use participatory design techniques to develop a
series of virtual instruments for use in live performance and then to examine closely
the experiences of musicians who use them.
An interaction design strategy which uses simulated physical models to mediate
between the sounds produced by acoustic instruments and computer generated
sounds and visuals has been developed. In this approach, a simple physical system
is modelled in software and characteristics of acoustic sounds are mapped to
forces and other parameters which affect the model. In response the model moves
in ways that are physically realistic. These movements are then used as parameters
to control video and audio synthesis.
Using a research approach which draws on action research, design science and
participatory design, a series of virtual instruments which use this interaction technique
were developed and used in live performances. A set of initial design criteria
which guided development were identified. In order to refine these criteria and better
understand the impact that using these virtual instruments has on musicians’ musicmaking,
a series of user studies were conducted. A number of expert musicians
used the virtual instruments and discussed their experiences. These sessions were
video-recorded, transcribed and analysed using grounded theory techniques.
The results of the study identified three modes of interaction with the virtual instruments:
instrumental, conversational and ornamental. Musicians interacting with
the virtual instruments in instrumental mode emphasise the importance of being in
control and being able to trust that the instrument will respond consistently. When
musicians use a virtual instrument ornamentally, they surrender detailed control of
the generated sound and visuals to the computer, allowing it to create audio-visual
layers that are added to the musicians’ sound. The more complex, (and difficult to
design for) conversational interaction involves the sharing of control between the musician
and the virtual instrument. The balance of power is in flux, allowing the virtual
instrument to talk back to the musician, reflecting and transforming the sonic input in
ways that move the performance in new musical directions.
The contributions of this thesis are therefore:-
a set of virtual musical instruments which use a unique interaction paradigm
in which simulated physical models mediate between live sounds produced on
acoustic instruments and computer generated sounds and visuals;
a theory of musician-virtual instrument interaction; and
a set of design criteria informed by practice and user studies.
Please use this identifier to cite or link to this item: