Multimodal interfaces in Virtual, Augmented, and Mixed Realities promise to become novel forms of Human-Computer Interaction, taking into account the requirements defined by the physical, cognitive, and perceptive skills of users. Central aspect of such interfaces is their interactive nature combined with simulated spatial 3D content and environments, paving way for enhanced embodiment, immersion, and presence. These interfaces comprehensively apply multimodality to the output as well as to the input side of closely-coupled human-computer systems. A variety of output channels for, e.g., visual, auditive, or haptics stimuli is complemented by multiple interaction paradigms based on direct manipulation, touch, speech and gesture, or even virtual agents. While each of the former I/O-related aspects in itself defines its own sets of requirements and methods to solve it, the highly interactive nature of such systems drastically complicates conceptual and technical solutions. This talk will introduce the state-of-the-art, from scientific prototypes to computer game technology and will highlight some ongoing developments. Along the way, we will point-out chances as well as critical aspects of this technology which has already begun to enter our everyday lives, from our living-rooms to future work places.


Marc studied mathematics and computer sciences at the University of Paderborn, the New York Institute of Technology and the Bielefeld University. After several accompanying years in the computer business, he received his PhD in 2001 in the area of multimodal–gesture and speech–interaction for Virtual Reality. He headed the AI & VR Lab at the Bielefeld University until 2007, became professor for media informatics at the University of Applied Sciences (HTW) in Berlin, and founded the Intelligent Graphics Group at Bayreuth University in 2009.

Marc holds the chair for human-computer interaction at Würzburg University since 2011. His work is interdisciplinary oriented towards human-computer interaction interconnecting real-time 3D graphics and simulation, virtual and augmented environments, Artificial Intelligence, and cognitive sciences.