How can we develop appropriate and natural forms of communication between humans and computers? In the future, computers will hardly be perceived as such, but will be seamlessly integrated into our living and working environments, e.g. in cyber-physical systems. How can people be effectively supported with their experiences, sensory organs, gestural means of expression and social needs?

Our work focuses on the systematic and fundamental research of different, preferably natural interaction modalities and their synergistic combination. We are researching gestural multi-touch interaction, digital pens in combination with digital paper, the efficient combination of touch and pen input, gestural interaction with hands, head and feet, gaze-supported interaction for remote displays, and tangibles. Our focus is on small and large interactive surfaces (mini-displays, smartphones, tablets, tabletops, high-resolution large wall displays) and their effective combination in multi-display environments.

We apply the developed techniques and principles in various application domains and projects with academic and industrial partners from Germany and abroad. We also investigate how modern user interfaces and interaction techniques can be used effectively in the fields of interactive information visualization, music informatics and semantic web.

Recent Research Projects

Preview for research project: Augmented Dynamic Data Physicalization Augmented Dynamic Data Physicalization Preview for research project: Gazing Heads Gazing Heads
Preview for research project: PMC-VIS: Interactive Visualization for Probabilistic Model Checking PMC-VIS: Interactive Visualization for Probabilistic Model Checking Preview for research project: PEARL: Physical Environment based Augmented Reality Lenses for In-Situ Human Movement Analysis PEARL: Physical Environment based Augmented Reality Lenses for In-Situ Human Movement Analysis
Preview for research project: Evonne: Visual Explanations of Logic and Numerical Reasoning Evonne: Visual Explanations of Logic and Numerical Reasoning Show all research projects

Recent Publications

  • Pixel Memories: Do Lifelog Summaries Fail to Enhance Memory but Offer Privacy-Aware Memory Assessments?

    Elagroudy, P.; Rzayev, R.; Machulla, T.; Le, H.; Dingler, T.; Lischke, L.; Clinch, S.; Ward, G.; Schmidt, A.;

    @inproceedings{Elagroudy2025pixelMemories,
       author = {Passant Elagroudy and Rufat Rzayev and Tonja-Katrin Machulla and Huy Viet Le and Tilman Dingler and Lars Lischke and Sarah Clinch and Geoffrey Ward and Albrecht Schmidt},
       title = {Pixel Memories: Do Lifelog Summaries Fail to Enhance Memory but Offer Privacy-Aware Memory Assessments?},
       series = {CHI '25},
       isbn = {979-8-4007-1394-1/25/04},
       numpages = {17},
       doi = {10.1145/3706598.3714145},
       url = {https://doi.org/10.1145/3706598.3714145},
       address = {New York, NY, USA},
       keywords = {lifelogging, recall, memory research, privacy, case study}
    }

  • A user fanning out several holographic documents in their hands and enlarging one of them.

    Documents in Your Hands: Exploring Interaction Techniques for Spatial Arrangement of Augmented Reality Documents.

    Luo, W.; Ellenberg, M.; Satkowski, M.; Dachselt, R.;

    @inproceedings{luo2025documents,
       author = {Weizhou Luo and Mats Ole Ellenberg and Marc Satkowski and Raimund Dachselt},
       title = {Documents in Your Hands: Exploring Interaction Techniques for Spatial Arrangement of Augmented Reality Documents},
       series = {CHI '25},
       isbn = {979-8-4007-1394-1/25/04},
       numpages = {22},
       doi = {10.1145/3706598.3713518},
       url = {https://doi.org/10.1145/3706598.3713518},
       address = {New York, NY, USA},
       keywords = {spatial layout, content organization, interaction design, user-centered design, Mixed Reality}
    }

  • ARbiter: Generating Dialogue Options and Communication Support in Augmented Reality.

    Méndez, J.; Satkowski, M.;

    @inproceedings{mendez2025arbiter,
       author = {Juli\'{a}n M\'{e}ndez and Marc Satkowski},
       title = {ARbiter: Generating Dialogue Options and Communication Support in Augmented Reality},
       series = {ACM CHI'25},
       numpages = {3},
       doi = {10.48550/arXiv.2503.05220},
       keywords = {Augmented Reality, Artificial Intelligence, Conversational User Interfaces, Real-time Dialogue, Language Models, AI assistance}
    }

  • Endomersion: An Immersive Remote Guidance and Feedback System for Robot-Assisted Minimally Invasive Surgery.

    Ellenberg, M.; Krug, K.; Fan, Y.; Krzywinski, J.; Dachselt, R.; Younis, R.; Wagner, M.; Weitz, J.; Rodriguez, A.; Just, G.; Bodenstedt, S.; Speidel, S.;

    @inproceedings{ellenberg2025endomersion,
       author = {Mats Ole Ellenberg and Katja Krug and Yichen Fan and Jens Krzywinski and Raimund Dachselt and Rayan Younis and Martin Wagner and J\"{u}rgen Weitz and Ariel Rodriguez and Gregor Just and Sebastian Bodenstedt and Stefanie Speidel},
       title = {Endomersion: An Immersive Remote Guidance and Feedback System for Robot-Assisted Minimally Invasive Surgery},
       series = {IEEE VR '25},
       numpages = {2},
       keywords = {Mixed Reality, Remote Guidance, Telestration, Minimally Invasive Surgery}
    }

  • Immersive Data-Driven Storytelling: Scoping an Emerging Field Through the Lenses of Research, Journalism, and Games.

    Méndez, J.; Luo, W.; Rzayev, R.; Büschel, W.; Dachselt, R.;

    @article{MLRBD-2025-ImmDataStoriesReview,
       author = {Juli\'{a}n M\'{e}ndez and Weizhou Luo and Rufat Rzayev and Wolfgang B\"{u}schel and Raimund Dachselt},
       title = {Immersive Data-Driven Storytelling: Scoping an Emerging Field Through the Lenses of Research, Journalism, and Games},
       doi = {10.1109/TVCG.2025.3531138},
       address = {New Jersey}
    }

  • Show all publications