Ben Fletcher-Watson

From stage to screen: adapting a children's theatre production into a digital toy

This photo essay explores the adaptation of the Scottish theatre for early years (TEY) production White into a transmedia digital toy. In 2013, Catherine Wheels Theatre Company commissioned an app developer, Hippotrix, to create an app for mobile tablet computers and smartphones inspired by the world of  White.

The paper outlines the process of creating White The App, including wire-framing, asset capture, sound recording, coding and prototyping. It also explores the impact of design decisions on dramaturgy and performativity, noting that digital media offer new possibilities for embracing non-linear storytelling while retaining key aspects of the live aesthetic.

Keywords: White, app development, iPad, theatre for early years

Introduction

White in performanceWhite in performance.
Image courtesy of Catherine Wheels Theatre Company, copyright Douglas McBride.

This paper explores the adaptation of Catherine Wheels’ highly successful production White into a mobile app for Apple’s iPad and iPhone. The theatrical version, created for children aged two to four and their families, has been performed more than 700 times around the world since its première at the Edinburgh Festival Fringe in 2010, and has been translated into several languages, including French, Swedish, Welsh (Gwyn, Cwmni’r Frân Wen) and Norwegian (Hvit, Barneteatret Vårt). White may be the most successful non-commercial theatre for early years (TEY) production to date, in terms of audience figures.

The iPad, Apple’s tablet computer, also emerged in 2010. In 2014, over 245 million tablet computers are projected to be sold worldwide (data from idc.com). In the USA, more than 25% of adults have purchased apps (an abbreviation of ‘applications’) for their children, and apps aimed at preschoolers now constitute a majority of education products in Apple’s App Store (Shuler et al., 2012). A key factor in the success of tablet computers with the under-fives is that ‘they cater for multiple competencies on a single portable device. Users can switch at will between activities which promote varying skills, from literacy to hand-eye coordination, from joint attention to imitative role play’ (Fletcher-Watson, 2013, p. 58).

In 2013, Catherine Wheels Theatre Company commissioned Hippotrix, a commercial app developer based in East Lothian, to create an app for mobile tablet computers inspired by the world of White. This paper outlines the process of creating White The App and explores the impact of design decisions on dramaturgy and performativity.

Brief

As producer Paul Fitzpatrick states, in seeking to create a digital artistic product, Catherine Wheels ‘wanted to provide an experience that would be creative as well as reflecting the artistic integrity of the show’ (Fitzpatrick, 2014). In addition, while the production of White is aimed at children from two to four, the developers were tasked with producing an app suitable for children from one to five, in order to increase appeal to families. The app was also to be playable on both iOS tablets (iPads) and smartphones (iPhones).

White The App was intended to complement the touring production, while also appealing to users who may never have seen it, the original narrative becoming a transmedia story. As Jenkins (2006, p.98) notes of other transmedia, ‘[e]ach franchise entry needs to be self-contained so you don’t need to have seen the film to enjoy the game, and vice versa. Any given product is a point of entry into the franchise as a whole’. Indeed, a discrete product may even follow a unique narrative path ‘to provide greater pleasure, excitement, and depth for fans’ (Laurel, 2013, p.183). The brief thus allowed Hippotrix to develop interactive scenarios derived from the mythology of White, but not copied directly from performed events.

A transmedia franchise consisting of an originator and its satellite derivatives can be considered as an ‘augmented product [with] features and benefits beyond what the target audience normally expects’ (Kotler and Scheff, 1997, p.193). This is an increasingly familiar model within the arts, and the augmentation process can be situated within the framework of Crealey’s (2003) strategies for minimising risks for consumers of culture, in particular the proposal for a system of theatrical product testing, allowing potential spectators to access early versions of performance comparable with product prototypes. This ‘try before you buy’ approach reduces financial and emotional risk to the consumer, as it allows them to experiment with the artistic product at minimal cost before choosing whether to engage more fully. In the arts, it is already common for ‘consumers [to] engage in risk-reducing (i.e. information-search) activities in order to reduce their perceived risk level (and therefore, their feelings of being uncomfortable)’ (Dowling and Staelin, 1994, p.121), such as reading reviews in advance. However, transmedia may in fact stimulate a greater interest in the originator product than typical risk-reducing activities, as ‘[r]eading across the media sustains a depth of experience that motivates more consumption’ (Jenkins, 2006, p.98).

Child playing with iPadChild playing with iPad.
Image courtesy of Hippotrix.

Digital toys

TEY has developed discrete forms, wholly distinct from theatre for older children, and arguably considerably more radical. The three commonest modes, albeit with considerable overlap, are interactive theatre (moments of pre-determined interaction or exchange between performer and audience), participatory theatre (performances wholly or in large part created by the audience) and promenade / immersive theatre (performances taking place in multiple sites, some of which may be virtual). It is possible to categorise existing digital formats within these modes: Wii and Kinect games1 tend to permit interactivity, allowing certain behaviours to be expressed within defined limits, moving towards a pre-determined objective; mobile apps for the very young, such as those developed by Toca Boca and Kapu, tend to be participatory in form, often deferring defined goals in favour of infinite repetition (Stoll, 2013). Augmented reality and Google Glass are excellent potential systems for immersive theatre, although this has not yet been explored in depth (one notable exception being Fish & Game’s iPad-based performance Alma Mater (2010) which imitated augmented reality within a real environment—by following a prescribed narrative path but lacking an explicit plot, the production provided an illusion of user control).

Just as a live performance is neither game nor story-text (although it can contain elements of both), so a theatrical app is not necessarily an ‘as-live’ video recording of the show, like Met Opera On Demand for the iPad or NT Live in cinemas (NESTA, 2011), nor a computer game with levels to be completed, nor an enhanced storybook with voiceover. The term ‘digital toy’ has come to be used to describe a more free-form playable scenario where the user is permitted to explore a virtual world (Lauwaert, 2009), arguably comparable to a participatory theatre experience2. Like a physical toy, the user can interact with the components however they choose. There are no rewards for success nor penalties for failure, and the experience can last for as long as desired. The non-competitive, non-didactic, exploratory nature of digital toys appealed to the developers as the closest fit with both Catherine Wheels’ live work and the capabilities of its audience.

Interaction design

Leiberman et al. (2009) have identified three examples of promising practice in digital design for very young children: media should be developmentally-appropriate, evidence-based and tested with children. This closely mirrors contemporary praxis within arts for early years, where productions are often inspired by specific developmental milestones, and / or rooted in child psychology, and almost always use invited audiences of children and caregivers to guide the devising process (Fletcher-Watson et al., 2014). Hippotrix’ first design consideration was therefore defined by the age range specified for the app. The available interactions (meaning the control mechanisms for gameplay permitted by the iPad’s touchscreen interface) were carefully considered, and a range of competitors’ products examined, to ensure that all interactions would be accessible regardless of the user’s age.

Early wireframe diagram of White The AppEarly wireframe diagram of White The App.
Image courtesy of Hippotrix.

Typical interactions include tap, swipe, tap-and-hold, pinch and multi-finger swipe. Rarer interactions may include rotate, speak (via the integral microphone), shake (usually to reset) or motion detection (via the integral front-facing camera). Fewer than 20% of the early years apps surveyed employed any interactions other than tap, swipe and tap-and-hold, probably due to the more complex motor skills required to pinch, rotate and so forth. The decision was therefore made to limit controls in White The App to these three basic movements.

In addition, the interactions were not designed to build in complexity nor to comprise a given sequence, as is common in more educationally-centred products. In keeping with the tenets of digital toys, the app was designed to allow infinite gameplay within scenes. TEY performances often feature duplicated sequences to enhance engagement, as in White, where the second half closely matches the first. However, they cannot repeat endlessly, whereas an app can permit immediate and constant repetition of any action, aiding understanding and laying the foundations for structured learning.

Development and user testing

The use of real objects was a key design decision for White The App from the pre-production stage: just as original illustrations are often preserved in e-books, so theatre apps can choose to accurately retain the look and feel of live performance by employing physical assets used on stage. Original production design is thus preserved, but the experience also reinforces the link to ‘liveness’, going some way to inverting Auslander’s claim that ‘live performances now emulate mediatized representations’ (2008, p.158). These are not animated cartoon versions of actors, props and scenery, but images of the real original objects themselves. Similarly, physics engines were employed to generate accurate reactions of objects as they are manipulated, bouncing off one another or stacking up (Millington, 2007).

Hippotrix achieved this photorealistic effect by the use of greenscreening (also known as chroma key photography), a simple way of isolating objects to allow them to be reused frequently throughout an app. Used for decades in film, animation, video games and television, greenscreening involves capturing an image of a specific object or person in front of a single-colour background (originally blue, but generally green since the advent of digital cameras). The background can then be removed in post-production and a new backdrop added (Wolf, 2003).

Chroma key in White The AppChroma key in White The App.
Image courtesy of Hippotrix.

The benefit of chroma key is that the edges of the image are extremely crisp and detailed, unlike the effect produced by manually cutting out objects in photo-editing programs such as Adobe Photoshop. White revolves around two key sets of objects: eggs and birdboxes. The eggs were simple to isolate, having smooth edges, but Shona Reppe’s award-winning design for each birdbox is a riot of detail—lace, wool, mirrors, pencils, shaggy carpet. Chroma key simplified the task of capturing such assets considerably.

Maquette for White by Shona ReppeMaquette for White by Shona Reppe.
Image courtesy of Catherine Wheels Theatre Company.

As one critic noted of the original production:

Great theatre design does not just illuminate its text, it adds to it. Shona Reppe's design and costumes for Catherine Wheels’ White went one step further again. She created a white world of such invention that it could have existed on its own (Dibdin, 2011, p.1).

The app aimed to create a ‘white world’ encompassing objects, backgrounds and characters that would appeal to users whether or not they had seen the production. However, the use of live actors in addition to original props was problematic. Aside from the issue of expensive rights for a performer’s likeness, the necessity to record multiple versions of all possible interactions to accommodate a user-defined dramaturgy would have exceeded the budget available for the project. In addition, the time required to greenscreen an entire video sequence, as opposed to a single image, would have been too great. This was solved by placing the user in the position of characters Cotton and Wrinkle, rather than as a new character interacting with them. Thus they do not appear in person, but instead the user becomes their avatar, immersed in the recreated environment of the scenery and props: ‘[i]n digital gaming, the player is… viewing his or her own actions—the actions one's avatars carry out can be seen on the screen’ (Vangsnes, 2009, p.31).

Andy Manley and Ian Cameron recording vocal parts for White The AppAndy Manley and Ian Cameron recording vocal parts for White The App.
Image courtesy of Hippotrix.

Another means of preserving a performative link to the original production was to use the original actors and devisers, Andy Manley and Ian Cameron, as voice artists for the app. They created the sounds of chattering eggs, creaking doors and the occasional line from their characters, Cotton and Wrinkle respectively. Similarly, composer Danny Krass was engaged to extend the brief tracks he had created for the live show into loops which could run behind the action for as long as required.

Via the development process, and in collaboration with the original production team, four scenes were selected for inclusion: the first depicts white eggs falling through cotton-wool clouds, with the occasional surprising appearance of a red egg; the second scene allows users to care for a number of eggs, applying sticking plasters, kisses, showers or woolly hats as necessary; the third sequence turns the seven birdhouses in which the eggs sleep into a playable piano; and the fourth reflects the production’s finale with a colourful party atmosphere. In each case, elements of the live show were mixed with new concepts to create a self-contained scenario. In addition, the narrative links between scenes were kept deliberately tenuous, allowing for multiple interpretations.

Testing the app with young children and caregiversTesting the app with young children and caregivers.

Within TEY, and indeed within the development process for the original production of White, testing with the target audience during rehearsals has emerged as an almost universal practice, unlike in adult theatre. App developers tend to conduct similar prototyping via mobile analytics testers such as TestFlight or HockeyApp, which also provide crash reports and options for feedback. White The App was tested in January and February 2014, with users across the required age range and their caregivers providing responses prior to release. The prototype app was presented to users in both domestic and nursery settings. Individual play and verbal feedback were recorded simultaneously on video, while carers were invited to submit comments via email. The testing phase had a significant impact on the app, from the timing of certain sequences to the addition of new assets. For example, in the third scene, younger users expressed frustration with the relatively small target areas which triggered music effects, so the decision was made to respond to taps anywhere on the screen with new sounds, such as an owl hooting. The birdhouses still played specific piano notes when touched, but the backdrop of the night sky also became playable. An iterative post-production process of bug-fixing, amending scenes based on user feedback, and streamlining was then used to complete the app, which launched in March 2014.

Problematising traditional modes of performativity

Virtual space and interactive video games offer new dramaturgical possibilities, while also drawing on and provoking the interactivity of the theatre event (Turner and Behrndt, 2008, p.198).

The app format allows users to explore performance in new ways which a live experience cannot facilitate. For example, White, unusually among TEY productions, employs a linear narrative, albeit one that is repeated; like Waiting for Godot, the second act mirrors the first closely, emphasising the impact that an outside force can have on a closed world where change is alien. The decision was made to move away from narrative in the app, instead presenting the user with agency to explore a series of discrete scenes with only tenuous connections to one another (and then perhaps only for users who have seen the production). As a digital toy, White The App rejects traditional forms of digital storytelling in favour of a repeatable (post)dramaturgy drawn from TEY: ‘when the user is allowed freedom of action the usual laws of linear expression drama no longer apply’ (Aarseth, 1997, p.138). Users may play in a single scene endlessly, or skip scenes entirely. It should again be noted however that the original production does not employ such a dramaturgy—the decision to fragment the narrative in the app was made due to the expanded age range required, from one to five years, in order to grant agency to the youngest users, even if they engage only in a single activity.

In addition, gameplay is defined and restricted by the allowable rules of the ‘closed world’ system—the user cannot speed up or slow down the descent of the eggs, for example, or carry out an action which has not been explicitly designed into the app (see for example Salen and Zimmerman 2006). Indeed, Huizinga’s (1955, pp.10-12) compendious study of play describes this necessary limiting of agency in numerous contexts:

All play moves and has its being within a play-ground marked off beforehand either materially or ideally, deliberately or as a matter of course... the stage, the screen, the tennis court, the court of justice, etc., are all in form and function playgrounds, i.e. forbidden spots, isolated, hedged around, hallowed, within which special rules obtain. All are temporary worlds within the ordinary world, dedicated to the performance of an act apart.

As in a physical playground, the user can elect to play with any item in any order, but they are limited to the items provided by the underlying code (Kline and Dyer-Witheford, 2003). This can usefully be compared to the post-performance session after White, when children are allowed to meet the actors and play with coloured confetti in the auditorium but may not enter the ‘forbidden spot’ of the stage to examine the props and set more closely. The app by contrast grants the opportunity to investigate these items at no risk to the child or the performance, although it does not grant agency to deconstruct the objects. The user’s interactions are purposely modelled on Cotton and Wrinkle’s actions, not those of an inquisitive child. Thus they may open doors or kiss eggs, but may not drop an egg to see what happens. Here, interaction is dramaturgically defined.

Excerpt of code from White The AppExcerpt of code from White The App.
Image courtesy of Hippotrix.

Johanson and Glow (2011, p.66) have critiqued digital storytelling as uninspiring and limiting a child’s imagination:

[Rose] Myers [of Windmill Theatre in Australia] finds that theatre plays a role in inspiring the imagination in a way that screen-based activities, with their use of computer-generated imagery, do not… [quotation from Myers] ‘we’re requiring you to work a lot harder because we can’t give you all the special effects. I think that nurtures children’s own creativity because it’s much closer to the way that kids themselves play games and create things, make up stories and act them out. So it engages children and shows them the possibilities’.

Lauwaert (2009, p.21) notes that comparisons between the physical and the digital tend to result in simplistic binary oppositions such as small-scale personal play versus massive online / social play or tactility versus ‘isolated, immobile and escapist play’. However, in the case of digital theatre, these binaries may become blurred—for a user who has seen the performance, the experience of interacting with familiar photorealistic assets (as opposed to computer-generated images) within an replayable dramaturgy not only reproduces the source materiality as faithfully as possible, but also allows the user to delve deeper into the narrative and aesthetic than was possible when spectating in the theatre: picking it apart and re-ordering it at will. For users who have not yet seen the live version, the scenes may inspire a unique personal narrative journey which moves beyond the fixed dramaturgy of the performance.

A more pertinent critique is offered by Plowman and McPake’s (2013, p.31) assessment of the limitations of digital interaction in e-books:

The touch screen and gestural interface, the portability and easy share-ability offer new dimensions of interactivity but… some apps simply reproduce tired versions of electronic books rather than exploit the affordances of the medium. Technological interactivity is meagre compared to human interaction... An electronic book that reads the words out one at a time or asks children to point to a picture with the stylus and then says ‘well done’ cannot simulate the experience of adult-child conversations.

The creation of an e-book app telling the story of White was abandoned early in pre-production in favour of the digital toy format, but the ‘meagre’ interactivity of digital narratives was observed within competitor products. Here, the decision to situate the user as an avatar of Cotton or Wrinkle not only delineates the dramaturgy of the app as noted above, but also transforms the user into what Ryan (2001, p.17) has called ‘the interactor [who] performs a role through… physical actions, thus actually participating in the physical production of the text’. The digital gestures of the child construct the identity of the pre-existing character without the need for the original actor to be present. Instead, the objects which represent their role are made available to the user to play with as they wish. The app can thus be said to trouble performativity, rejecting a diegetic narrativism equivalent to ‘reading out the words’ in favour of a mimetic digital presence.

It is interesting briefly to consider the ways in which performativity in the Butlerian sense is also blurred by this transformation from spectator to interactor. The visually-gendered stage action (both protagonists are male) of the live version becomes more fluid in the app, where the interactor may be female or male, or indeed a mixture of genders if several children are playing together. Previous spectators may remember that they are embodying a male avatar, but users new to the cultural product of White are given no hints as to their digital gender. Similarly, their actions lack an overtly gendered frame—opening a door is neither stereotypically male nor female, for example—so the user is left to construct meaning without the semiotic certainties granted by spectatorship: before or after they engage with the app, a child may or may not see two men act out the story of White in a theatre, and this may (or may not) impact on the performativity of the digital act.

Returning to Plowman and McPake’s appraisal of current technologies, their challenge to developers to pursue a more profound interactivity is in fact extended to those products (generally but not exclusively educational) which are used by a child alone. The use of electronic media of all kinds, from DVDs to iPads, as ‘digital babysitters’ has received considerable media attention (Cocozza, 2014; Donnelly, 2013; Dredge, 2013), but there is little to justify this beyond anecdote (Wood, 2008). White The App, like the production, is designed as a shared experience, for children and carers to enjoy together. It is possible to compare shared experiences with the pedagogic model of guided interaction (Plowman and Stephen, 2007): in both cases, the involvement of an adult is intended to elicit deeper engagement in a given activity by a child; in both cases, the dyadic bond is strengthened by mutual interest in the activity—the child appreciates the attention that is being paid to them, and the adult appreciates the child’s focus on the activity. Studies examining infant-directed television programmes have found that children’s attention and responsiveness are significantly influenced by the extent of parental involvement (Barr et al., 2008) and it is possible that engagement with digital toys would be similarly affected.

Indeed, unlike the live performance, the app version permits or even encourages conversation. It must be remembered that a live experience generally requires silence from its audience—children are expected to keep quiet until the resolution of the performance, negating the possibility of conversations or questions during a scene to aid understanding. An app experience is communal and communicative, allowing children to comment meta-narratively and leaving space for adults to pose questions, praise or guide.

In both physical and digital scenarios, shared focus is a stepping-stone towards eventual solo interaction. Once a child has mastered the basic movements required, the adult can observe without needing to assist at any point.

Gameplay image from White the App
Gameplay image from White the App
Gameplay image from White the App
Gameplay image from White the AppGameplay images from White The App.
Images courtesy of Hippotrix.

Conclusions

This paper describes the conversion of a live performance for young children into a digital experience on a tablet. The process outlined above, from initial brief to release of the final product, will be familiar to manufacturers and designers, but the role of the child as avatar or interactor, as well as the parent as facilitator, troubles theatre practitioners’ traditional conceptions of spectatorship.

More research is needed to interrogate fully the impact of tablet-based theatrical experiences on dramaturgy, spectatorship and performativity. In particular, if transmedia products such as White The App are to gain acceptance as being complementary or comparable to live theatre, it will be essential to address the dominant media narrative challenging children’s tablet use (see for example Jones 2011; Palmer 2011; Ward 2013), which often describes tablets as ‘digital babysitters’ and fails to draw distinctions between screen time devoted to television or apps.

As a complement to the live performance, the app version contains scenes and objects which will be immediately familiar to spectators, while simultaneously extending the mythology of the production by highlighting previously peripheral moments, such as the arrival of the eggs from above. For children who have not yet encountered the stage version, White The App is an open-ended story which they can construct at leisure, sharing the experience with a parent or sibling until they feel comfortable enough to roam on their own. It may encourage newcomers to attend the live production if they seek greater depth to the experience, or even prompt repeat visits to the theatre by fully-engaged spectators.

It is therefore clear that transmedia can offer theatre companies new means to monetise existing artistic properties and to attract new audiences for whom live theatre is not yet a regular pastime. Whether the medium of theatre apps for very young children has longevity is more difficult to predict. However, as a new form of engagement with theatre for very young children, translating a live 3-D performance into a replayable 2-D digital toy, it represents a bold step into the digital world for an art form which prides itself on ‘liveness’. Adults have shown remarkable willingness to engage with remotely viewed or non-present theatre, such as NT Live; perhaps tablet computers can provide babies and toddlers with similar access to the more interactive genre of theatre for early years.

Publicity image for White The AppPublicity image for White The App.
Image courtesy of Hippotrix.

Notes

  1. The Wii is Nintendo's motion-detecting video game console, released in 2006. The Kinect is a motion-sensing input device launched by Microsoft in 2010, and designed to connect to XBox 360, XBox One or PC. []
  2. ‘Digital toy’ sometimes also refers to physical toys with a digital interactive element, such as Sony’s robot dog AIBO (see, for example, Hsu et al. 2007). The varying terms ‘digital toy’, ‘virtual toy’ and ‘electronic toy’ remain contested.[]

This research is supported by the ESRC Capacity Building Cluster, ‘Capitalising on Creativity’, grant #RES 187-24-0014, and with the assistance of Hippotrix and Catherine Wheels Theatre Company.

References

Ben Fletcher-Watson is currently studying for a PhD in drama at the Royal Conservatoire of Scotland and the University of St Andrews, supported by an ESRC CASE Studentship. His research examines contemporary Scottish practice in theatre for early years. He has published articles in journals including Youth Theatre Journal and Research in Drama Education, and is Co-editor of the Scottish Journal of Performance. His wider research interests include autism-friendly performance, theatre for foetuses and the history of pantomime. He serves on the Executive of the Theatre and Performance Research Association (TaPRA) and is an ASSITEJ Next Generation Artist.