The first Screen installation I helped develop was for Siggraph 2007, specifically for the newly constructed StarCAVE system at UCSD. The cave is a 15 faced, fully immersive VR environment. The entire video display is stereoscopically projected, thus giving 3D video with glasses. A head tracking system provides locative information. A 15.1 channel speaker system augments the video component, providing a full spherical sound stage. As Screen plays, the audience interaction triggers sound events that correspond to text ripping, breaking, and being hit, as well as ambient sound events. Using ambisonic processing techniques extremely accurate spatialization was achieved even in this resonant structure.

A second installation of Screen was constructed at the Beale Center at UCIrvine. This installation was only for a single stereoscopic screen, and a four channel spatialization. However, the previous software version allowed an easy scalability to this reduced setup with minimal overhead.

Screen was created in the “Cave,” a room-sized virtual reality display. It begins as a reading and listening experience. Memory texts appear on the Cave’s walls, surrounding the reader. Then words begin to come loose. The reader finds she can knock them back with her hand, and the experience becomes a kind of play — as well-known game mechanics are given new form through bodily interaction with text. At the same time, the language of the text, together with the uncanny experience of touching words, creates an experience that doesn’t settle easily into the usual ways of thinking about gameplay or VR. Words peel faster and faster, struck words don’t always return to where they came from, and words with nowhere to go can break apart. Eventually, when too many are off the wall, the rest peel loose, swirl around the reader, and collapse. Playing “better” and faster keeps this at bay, but longer play sessions also work the memory text into greater disorder through misplacements and neologisms. While the discussion of the relationship between games and literary forms is longstanding, Screen uses text as play material in a way that this discussion has not previously explored.

In addition to creating a new form of bodily interaction with text through its play, Screen moves the player through three reading experiences — beginning with the familiar, stable, page-like text on the walls, followed by the word-by-word reading of peeling and hitting (where attention is focused), and with more peripheral awareness of the arrangements of flocking words and the new (often neologistic) text being assembled on the walls. Screen was first shown in 2003 as part of the Boston Cyberarts Festival (in the Cave at Brown University) and documentation of it has since been featured at The Iowa Review Web, presented at SIGGRAPH 2003, included in Alt+Ctrl: a festival of independent and alternative games, published in the DVD magazines Aspect and Chaise, as well as in readings in the Hammer Museum’s HyperText series, at ACM Hypertext 2004, and in other venues.

Mention:

• Grand Text Auto
• Kotaku

Installations:

• August 5-9, 2007. Siggraph Conference. UCSD StarCAVE. San Diego, CA.
• October 15 – December 15, 2007. Grand Text Auto. UC Irvine. Irvine, CA.