Interaction as Performance (Proc. CHI’05, ACM Press 2005)
Steve Benford & Gabriella Giannachi
The goal of this paper is to discuss about the strong relationship between interaction design, theatre and performance to form a conceptual framework as well as a language to express the possible way of computation that can be involved into performative experiences for the practitioners and researchers. Furthermore, the goal is to create a boundary object that might sit at the intersection of HCI and theatre and performance studies. This paper documents the mixed-reality performance, an emerging theatrical genre, which is intended to convey two major ideas: creating experience which will mix rich and complex real and virtual worlds; and combining live performance of participants in interactive digital media.
A series of landmark mixed-reality performances by various artists are discussed and analysed for further discussion.
Desert Rain (1997), based on first Gulf War, provides a great example of mixtures of real and virtual worlds, forms a hybrid theatrical set. It uses series of “rain curtains” (projection screens made of fine water spray through which participants can pass). Ethnographic studies showed how the artists orchestrated the experience to create key climatic moments in the performance.
The following example shows the hybrid city streets model where online players are chased by actors equipped with GPS along with streaming live audio to catch them. The researcher’s study revealed how the experience needed to accommodate so-called seams (in the underlying technical infrastructure.
Similar to previous work, Uncle Roy All Around You (2003), connected an online virtual world to city streets but focusing on how a performance might exploit the ambiguity of relationships between street players, online players and public spectators. The street players were engaged with props, locations and actors in the city. Ethnographic studies expressed the significant challenges of orchestrating a distributed mobile experience on city streets.
To explore the nature of mobile engagement furthermore, researchers worked with another mixed reality performance called “Rider Spoke (2007) where cyclists explored a city at night, recording a series of stories, leaving them at key locations, and then listening to the stories of others.
Rider Spoke (2007)
To explore complex temporal relationships between real and virtual world a text-messaging adventure game for mobile phones called “Day of the figurines (2006) took a different tack. Ethnographic studies showed the challenges of managing highly episodic engagement, especially how participants had to be responsible for interruption and changing pattern of phone use with other people like family, friends and colleagues.
This study focused on the multilayered nature of the instructions which managed to address the four different aspects of location, sequence of actions, public comportment and relationships to content.
The role of HCI researchers is not only to develop interfaces and supporting technologies for the participants but also to create authority and orchestration tools for the artists. Another major role is to study the participants’ experience that will provide “thick description” of experience from participant’s point of view. These studies have informed theoretical work through a series of conceptual frameworks that generalise key aspects of performative interaction for the wider HCI audience. Examples are:
- Potential benefits of ambiguity in interface designing that produce reflection and interpretation
- Seamful design
- Approaches to design the spectator experience
- Moulding discussions of how experiences are framed
First observation of the researchers is that the mixed-reality performances tend to be inherently hybrid in their structure as they are combination multilayered timescales and participants with different roles to integrate a diverse form of interface into a single experience. According to the arguments of the researchers, such task can be described in terms of three fundamental kinds of trajectory:
- Canonical Trajectory: Plan for the experience using different subjective forms such as scripts, set design, media and code designs, stage management instructions. Multiple canonical trajectory may have branching and rejoining structures.
- Participant Trajectory: Actual path that a particular participant makes through a particular instance of the performance. It may diverge from pre-planned canonical trajectories as participant makes individual choices.
- Historic Trajectory: Documentation of the experiences providing the reflections and retelling. Although being a vital role, it has often been neglected in the mixed-reality performance scenario.
Introducing trajectory forwards us to some controversial questions. Does the notation of trajectories have wider relevance to other domains of HCI? Studying and analysing different cultural experiences such as visits to museum, galleries and theme parks or playing pervasive games which share many of the characteristics of mixed-reality performance, researchers of this paper argues that it has relevance. They suggested that HCI needs to consider how a user experience expands across many Interface spaces, timescales and roles. Mixed-reality performances provides a concrete projection of trajectories which is considered to be the glimpse of future user experience.