Summary

We envision computer mediated collaborative performances where actors at physically remote locations can interact and coordinate their actions as if they are next to each other on the same stage or in the same room. Through various means, including audio, video and animations, each actor has a remote presence at one or several remote stages. We are interested in how to mask the effects on a performance caused by unavoidable delays and by distance. This talk describes a system, MultiStage, doing this for the visual side of a remote presence. At each stage four 3D cameras tiled back to back for an almost 360 degree view continuously record actors. The system processes the recorded data on-the-fly to discover actions by actors that it should react to, and it streams data about actors and their actions to remote stages where each actor is represented by a remote presence, a visualization of the actor. The visualizations are rendered on separate projectors or on a wall-sized high-resolution display wall. The remote presence will, depending upon the gestures detected, determine if it should change the visual representation of the actor to better project across distance what the actor is doing. The subsystem, Global Interaction Space, letting actors issue commands to MultiStage through gestures is described in some detail, as is its generalized applicability in letting a user efficiently control and coordinate multiple computers in a room. When the remote presences because of unavoidable network and processing delays lag behind enough to make it awkward for the actors to interact and it becomes noticeable to an audience, the system applies various techniques to hide this, including switching rapidly to a pre-recorded video or doing on-the-fly animations. The prototype currently scales across the Internet to three stages, and comprises in total 15 computers, 12 cameras, several projectors, and a 28-tile 22Mpixel display wall.

Vita

Otto Johan Anshus is a professor of computer science at the Department of Computer Science, University of Tromsø, Norway, and a professor at large at the Department of Informatics, University of Oslo, Norway. He has spent four years at the Department of Computer Science, Princeton University, as a visiting professor. He is a graduate of the University of Tromsø. He is heading the High Performance Distributed Systems group at the Department of Computer Science, University of Tromsø. Research interests include distributed and parallel systems, operating systems, systems support for human-computer interaction, small region, high-resolution interactive weather forecasting, energy efficient architectures, and Big Data processing and storing. The research methodology is systems research with extensive prototyping and xperimentation.