The performance of The Reception at CITRIS’s holiday party was more than just great entertainment; it was research. Find out how dancers and scientists are using tele-immersion to advance the state of their art and technology.
Guests at CITRIS’s holiday gala in December were treated to a unique and fascinating performance. Mingling in the Heart Memorial Mining Building lobby, they watched on a large projection screen as dancers in labs at UC Berkeley and the University of Illinois, Urbana-Champaign, came together to interact in a virtual three-dimensional environment.
Behind the scenes of the performance, a veritable ballet of advanced technology was taking place. As dancers moved in the 4-by-4 foot tele-immersion lab at UC Berkeley, 48 cameras arranged in 12 stereo clusters captured their movements from all sides. Twelve computers (one per cluster) then reconstructed the images and sent them via Internet II to yet another computer, where they were combined with images of a single dancer at the University of Illinois, captured using the same method, and rendered in 3D on a virtual stage.Behind the scenes of the performance, a veritable ballet of advanced technology was taking place. As dancers moved in the 4-by-4 foot tele-immersion lab at UC Berkeley, 48 cameras arranged in 12 stereo clusters captured their movements from all sides. Twelve computers (one per cluster) then reconstructed the images and sent them via Internet II to yet another computer, where they were combined with images of a single dancer at the University of Illinois, captured using the same method, and rendered in 3D on a virtual stage.
The result? The dancers were able to perform together in semi-real time in spite of being thousands of miles apart. It is this ability for geographically distant people to meet in a third, new environment and interact in the same ways they would face-to-face—whether dancing or simply having a conversation—that differentiates tele-immersion from its predecessors like Virtual Reality.
Tele-Immersion allows dancers thousands of miles apart to perform together.
The event, co-sponsored by Humanities, Arts, Science, and Technology Advanced Collaboratory (HASTAC), was unprecedented not only because it was the first to use tele-immersion technology in a live performance, but because the performance itself was research. It is a model that is being pioneered by the Resonance Project, recently founded by Lisa Wymore with the aim of bringing together scientists and artists to explore the creative, communicative, and technological possibilities of emerging interactive media—tele-immersion in particular.
“It is an amazing reconstruction of real-time images that is not animated in any way. It has a fullness and depth to it that is really striking,” says Wymore, an assistant professor in Berkeley’s Department of Theater, Dance, and Performance Studies. Wymore organized the performance and choreographed a subsequent dance piece titled The Reception, which integrated themes from the technology such as multiple viewing points and screen-based representations of the self.
From the early portrayals of dance in film to the current use of motion-capture technology by major choreographers like Merce Cunningham, where technology leads, dance follows. “Dance artists have always been experimenting with how the body can be seen in time and space, how bodies can move in new ways and be seen in new ways,” Wymore explains.
Wymore sees tele-immersion as a giant leap forward for her art. The multiple viewing points and three-dimensionality of the technology not only alter how artists like herself view and understand the body, but also open up entire new ways of moving and interacting.
“When you move in the technology, your body can literally dissipate and atomize or fragment. It creates this beautiful effect, almost like Impressionist paintings,” Wymore explains, adding that on a virtual stage, bodies can even pass through one another
These results are actually glitches in the still-developing technology, which currently lacks haptic feedback and has difficulty processing fast movements. For Wymore, however, “it is an unexpected outcome that is very beautiful.”
For researchers, these same “accidents” provide useful feedback to improve their system. If the CITRIS event gave dance artists like Wymore a chance to experiment with a new way of moving in time and space, for computer science professors Ruzena Bajcsy of Berkeley and Klara Nahrstedt of Illinois, it was a chance to test a technology that incorporates several sub-disciplines and that Bajcsy describes as “at the edge of current capabilities.”
The Resonance Project explores the possibilities for interaction between science and the arts.
“It’s not perfect, but with this performance we have shown it can be done,” Bajcsy says of tele-immersion. Based on what she and her team learned from the holiday party performance, they have closed the lab for two months to further calibrate their system. They hope, with improvements, the tele-immersion system will be able to acquire and reconstruct more quickly as well as better recognize and process photometrics such as illumination and reflections.
They are also raising additional funding. The labs at Berkeley and Illinois were built and are operated with a grant from the National Science Foundation, but Bajcsy says more money is needed to advance the state of the art and the technology of tele-immersion.
When the lab reopens in late March, the stage will be set for Act II of the project. On April 22, the Resonance Project will put on another performance (dates and other details below) that Wymore hopes will feature more interaction with the technology, possibly even a duet between two dancers in separate labs. Working in tandem, tele-immersion researchers will be able to see whether their hard work has paid off and to learn more for future iterations of their system.
In this ongoing dance between research and performance, it seems there is always one more step to learn.
Upcoming performances of The Reception:
April 20, 21, 27, 28 at 8pm
April 22*, 29 at 2pm
Zellerbach Playhouse
*The April 22 performance will be followed by a post-performance discussion led by N. Katherine Hayles, author of Being Here: Presence/Remote Presence within Live and Media Based Performance.
For more information:
UC Berkeley’s Tele-Immersion Lab
UC Berkeley Department of Dance, Theater & Performance Studies Spring 2007 schedule