Overview

This thesis examines spatial interaction for immersive mixed-reality visualizations.
It investigates three core research questions:

  1. What are promising types of immersive mixed-reality visualizations, and how can advanced interaction concepts be applied to them?
  2. How does spatial interaction benefit these visualizations, and how should such interactions be designed?
  3. How can spatial interaction in these immersive environments be analyzed & evaluated?

To address the first question, the thesis examines how various visualizations, such as 3D node-link diagrams and volume visualizations, can be adapted for immersive mixed-reality settings and how they stand to benefit from advanced interaction concepts.
Regarding the second question, it shows how spatial interaction in particular can help to explore data in mixed reality. There, it provides a look into spatial device interaction in comparison to touch input, the use of additional mobile devices as input controllers, and the potential of transparent interaction panels.
Finally, to address the third question, this thesis presents research on how user interaction in immersive mixed-reality environments can be analyzed directly in the original, real-world locations, and how this can provide new insights. Overall, with this research, the thesis contributes interaction and visualization concepts, software prototypes, and findings from several user studies on how spatial interaction techniques can support the exploration of immersive mixed-reality visualizations.

Download

  • Spatial Interaction for Immersive Mixed-Reality Visualizations

    Spatial Interaction for Immersive Mixed-Reality Visualizations

    Büschel, W.

    Qucosa - Technische Universität Dresden,2023.

    @phdthesis{fixme,
       author = {Wolfgang B\"{u}schel},
       title = {Spatial Interaction for Immersive Mixed-Reality Visualizations},
       year = {2023},
       month = {4},
       location = {Dresden},
       url = {https://nbn-resolving.org/urn:nbn:de:bsz:14-qucosa2-857141},
       publisher = {Qucosa - Technische Universit\"{a}t Dresden}
    }

    List of additional material

    Dissertation

The dissertation document is publicly available at QUCOSA.

Research Projects

Spatial Interaction for 3D Data Visualization with Mobile Devices

In this study, spatial interaction and touch interaction for 3D visualizations on mobile devices were examined. It investigated task performance and user preferences for different basic tasks. Furthermore, this project also describes interaction patterns identified during the analysis of the camera trajectories recorded during the study.

  • @inproceedings{Bueschel2017_3DInfoVis,
       author = {Wolfgang B\"{u}schel and Patrick Reipschl\"{a}ger and Ricardo Langner and Raimund Dachselt},
       title = {Investigating the Use of Spatial Interaction for 3D Data Visualization on Mobile Devices},
       booktitle = {Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces},
       series = {ISS '17},
       year = {2017},
       month = {10},
       isbn = {978-1-4503-4691-7},
       location = {Brighton, United Kingdom},
       pages = {62--71},
       numpages = {10},
       doi = {10.1145/3132272.3134125},
       url = {https://doi.org/10.1145/3132272.3134125},
       acmid = {3134125},
       publisher = {ACM},
       address = {New York, NY, USA},
       keywords = {3D Data Visualization, Immersive Visualization, Mobile Devices, Spatial Input, Tangible Displays}
    }

    List of additional material

    Video

Mobile-Assisted Exploration of 3D Data Spaces in AR

Building on the results of the previous project, this project is a study on the smartphone-assisted exploration of 3D data spaces in head-mounted augmented reality. Similar to the previous study, it compares interaction techniques with spatial components (i.e. moving a smartphone as a means of input) with more traditional on-phone touch input. However, this project concentrates on an AR setup using a head-mounted display. Furthermore, the project explicitly looks into combinations of touch and spatial interaction, also considering the degrees of spatiality, simultaneity, and guidance of the resulting compound techniques

  • Investigating Smartphone-based Pan and Zoom in 3D Data Spaces in Augmented Reality

    Investigating Smartphone-based Pan and Zoom in 3D Data Spaces in Augmented Reality

    Büschel, W.; Mitschick, A.; Meyer, T.; Dachselt, R.

    In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services.MobileHCI ’19, Taipei, Taiwan.ACM,page 1-13,2019.10.1145/3338286.3340113

    @inproceedings{bueschel2019,
       author = {Wolfgang B\"{u}schel and Annett Mitschick and Thomas Meyer and Raimund Dachselt},
       title = {Investigating Smartphone-based Pan and Zoom in 3D Data Spaces in Augmented Reality},
       booktitle = {Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services},
       series = {MobileHCI ’19},
       year = {2019},
       month = {10},
       isbn = {9781450368254},
       location = {Taipei, Taiwan},
       pages = {1--13},
       numpages = {13},
       doi = {10.1145/3338286.3340113},
       url = {https://doi.org/10.1145/3338286.3340113},
       publisher = {ACM},
       address = {New York, NY, USA},
       keywords = {Augmented Reality, 3D Navigation, 3D Data Exploration, Pan \& Zoom, Interaction Techniques, Immersive Visualization}
    }

    List of additional material

    Video, Slides

AR Graph Visualizations

This project explores data encoding using the edges of 3D AR node-link diagrams. It highlights how immersive visualizations open up opportunities for specialized visualizations. The project shows how the special properties of stereoscopic presentation in AR and data exploration by physical navigation inform the design of 3D graphs. It compares several basic edge variants in a user study regarding task performance and subjective scores. While this project is mainly focused on the visuals, it also examined how users made use of the physical space during the study

Transparent Tablets for Head-Mounted AR

This project examines the use of transparent, spatially tracked tablets as tangible controllers for head-mounted AR. This effectively brings the concept of a tangible magic lens for immersive environments to HMD-based AR. This research also draws from previous projects already examined additional mobile devices in AR for 3D data space exploration. This project resulted in the open research platform CleAR Sight for transparent tablet input in AR. It presents interaction and visualization concepts based on three use case applications, and describes expert feedback, which was collected through hands-on sessions.

Mixed Reality Interaction Analysis

This project presents the concept and implementation of the MIRIA application and toolkit for the mixed-reality visualization and analysis of spatial interaction data from immersive and multi-display environments. This research shows how interaction data from such immersive systems can effectively be visualized in-situ, i.e., in the same real-world context from which the data originates. This context can inform the information needs of the users, and incorporating it into the analysis process may often prove beneficial. This project is in part based on earlier experiences with the analysis of study data from previous projects.

Further Publications

In addition to the main research projects of the PhD thesis, a variety of further publications, workshops contributions and accompanying demonstrations were published:

  • @inproceedings{fixme,
       author = {Wolfgang B\"{u}schel and Katja Krug and Konstantin Klamka and Raimund Dachselt},
       title = {Demonstrating CleAR Sight: Transparent Interaction Panels for Augmented Reality},
       booktitle = {Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems},
       year = {2023},
       month = {2},
       isbn = {978-1-4503-9422-2/23/04},
       location = {Hamburg, Germany},
       pages = {1--5},
       numpages = {5},
       doi = {10.1145/3544549.3583891},
       publisher = {ACM}
    }

    List of additional material

    Video, Preview

  • @inproceedings{langner:2021:marvis,
       author = {Ricardo Langner and Marc Satkowski and Wolfgang B\"{u}schel and Raimund Dachselt},
       title = {MARVIS: Combining Mobile Devices and Augmented Reality for Visual Data Analysis},
       booktitle = {Proceedings of the 2021 ACM Conference on Human Factors in Computing Systems},
       year = {2021},
       month = {5},
       isbn = {978-1-4503-8096-6/21/05},
       location = {Yokohama, Japan},
       numpages = {17},
       doi = {10.1145/3411764.3445593},
       publisher = {ACM},
       address = {New York, NY, USA},
       keywords = {data visualization, mobile devices, head-mounted augmented reality, cross-device interaction, data analysis, mobile data visualization, augmented displays, immersive analytics}
    }

  • Grand Challenges in Immersive Analytics

    Grand Challenges in Immersive Analytics

    Ens, B.; Bach, B.; Cordeil, M.; Engelke, U.; Serrano, M.; Willett, W.; Prouzeau, A.; Anthes, C.; Büschel, W.; Dunne, C.; Dwyer, T.; Grubert, J.; Haga, J.; Kirshenbaum, N.; Kobayashi, D.; Lin, T.; Olaosebikan, M.; Pointecker, F.; Saffo, D.; Saquib, N.; Schmalstieg, D.; Albers Szafir, D.; Whitlock, M.; Yang, Y.

    In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.CHI '21, Yokohama, Japan.ACM,2021.10.1145/3411764.3446866

    @inproceedings{fixme,
       author = {Barrett Ens and Benjamin Bach and Maxime Cordeil and Ulrich Engelke and Marcos Serrano and Wesley Willett and Arnaud Prouzeau and Christoph Anthes and Wolfgang B\"{u}schel and Cody Dunne and Tim Dwyer and Jens Grubert and Jason H. Haga and Nurit Kirshenbaum and Dylan Kobayashi and Tica Lin and Monsurat Olaosebikan and Fabian Pointecker and David Saffo and Nazmus Saquib and Dieter Schmalstieg and Danielle Albers Szafir and Matthew Whitlock and Yalong Yang},
       title = {Grand Challenges in Immersive Analytics},
       booktitle = {Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems},
       series = {CHI '21},
       year = {2021},
       month = {05},
       isbn = {978-1-4503-8096-6/21/05},
       location = {Yokohama, Japan},
       numpages = {17},
       doi = {10.1145/3411764.3446866},
       publisher = {ACM},
       address = {New York, NY, USA}
    }

    List of additional material

  • Challenges in Collaborative Immersive Visualization

    Challenges in Collaborative Immersive Visualization

    Büschel, W.; Eckert, G.; Dachselt, R.

    In CHI 2020 Workshop on Immersive Analytics: Envisioning Future Productivity for Immersive Analytics.CHI '20, Honolulu, Hawaii, USA.page 1-6,2020.

    @inproceedings{fixme,
       author = {Wolfgang B\"{u}schel and Georg Eckert and Raimund Dachselt},
       title = {Challenges in Collaborative Immersive Visualization},
       booktitle = {CHI 2020 Workshop on Immersive Analytics: Envisioning Future Productivity for Immersive Analytics},
       series = {CHI '20},
       year = {2020},
       month = {4},
       location = {Honolulu, Hawaii, USA},
       pages = {1--6},
       numpages = {6}
    }

    List of additional material

  • Interaction for Immersive Analytics

    Interaction for Immersive Analytics

    Büschel, W.; Chen, J.; Dachselt, R.; Drucker, S.; Dwyer, T.; Görg, C.; Isenberg, T.; Kerren, A.; North, C.; Stuerzlinger, W.

    In K. Marriott et al. (Eds.), Immersive Analytics.Springer International Publishing,page 95-138,2018.10.1007/978-3-030-01388-2_4

    @inbook{fixme,
       author = {Wolfgang B\"{u}schel and Jian Chen and Raimund Dachselt and Steven Drucker and Tim Dwyer and Carsten G\"{o}rg and Tobias Isenberg and Andreas Kerren and Chris North and Wolfgang Stuerzlinger},
       editor = {Kim Marriott and Falk Schreiber and Tim Dwyer and Karsten Klein and Nathalie Henry Riche and Takayuki Itoh and Wolfgang Stuerzlinger and Bruce H. Thomas},
       title = {Interaction for Immersive Analytics},
       booktitle = {Immersive Analytics},
       series = {Lecture Notes in Computer Science},
       volume = {11190},
       year = {2018},
       month = {10},
       isbn = {978-3-030-01388-2},
       pages = {95--138},
       doi = {10.1007/978-3-030-01388-2_4},
       url = {https://doi.org/10.1007/978-3-030-01388-2_4},
       publisher = {Springer International Publishing},
       address = {Cham}
    }

    List of additional material

  • Investigating Link Attributes of Graph Visualizations in Mobile Augmented Reality

    Investigating Link Attributes of Graph Visualizations in Mobile Augmented Reality

    Büschel, W.; Vogt, S.; Dachselt, R.

    In CHI 2018 Workshop on Data Visualization on Mobile Devices.MobileVis ’18, Montreal, QC, Canada.2018.

    @inproceedings{Bueschel2018b,
       author = {Wolfgang B\"{u}schel and Stefan Vogt and Raimund Dachselt},
       title = {Investigating Link Attributes of Graph Visualizations in Mobile Augmented Reality},
       booktitle = {CHI 2018 Workshop on Data Visualization on Mobile Devices},
       series = {CHI '18},
       year = {2018},
       month = {4},
       location = {Montreal, QC, Canada}
    }

    List of additional material

    Workshop Website

  • Foldable3D: Interacting with 3D Content Using Dual-Display Devices

    Foldable3D: Interacting with 3D Content Using Dual-Display Devices

    Büschel, W.; Reipschläger, P.; Dachselt, R.

    In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces.ISS '16, Niagara Falls, ON, Canada.ACM,page 367-372,2016.10.1145/2992154.2996782

    @inproceedings{fixme,
       author = {Wolfgang B\"{u}schel and Patrick Reipschl\"{a}ger and Raimund Dachselt},
       title = {Foldable3D: Interacting with 3D Content Using Dual-Display Devices},
       booktitle = {Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces},
       year = {2016},
       month = {11},
       isbn = {978-1-4503-4248-3},
       location = {Niagara Falls, ON, Canada},
       pages = {367--372},
       numpages = {6},
       doi = {10.1145/2992154.2996782},
       url = {https://doi.org/10.1145/2992154.2996782},
       acmid = {2996782},
       publisher = {ACM},
       address = {New York, NY, USA},
       keywords = {3d interaction, augmented reality, dual-display device, head-coupled perspective, virtual reality}
    }

    List of additional material

    Poster

  • Improving 3D Visualizations: Exploring Spatial Interaction with Mobile Devices

    Improving 3D Visualizations: Exploring Spatial Interaction with Mobile Devices

    Büschel, W.; Reipschläger, P.; Dachselt, R.

    In ISS 2016 Workshop Immersive Analytics: Exploring Future Interaction and Visualization Technologies for Data Analytics.ISS '16, Niagara Falls, ON, Canada.ACM,2016.10.1145/3009939.3009949

    @inproceedings{fixme,
       author = {Wolfgang B\"{u}schel and Patrick Reipschl\"{a}ger and Raimund Dachselt},
       title = {Improving 3D Visualizations: Exploring Spatial Interaction with Mobile Devices},
       booktitle = {ISS 2016 Workshop Immersive Analytics: Exploring Future Interaction and Visualization Technologies for Data Analytics},
       year = {2016},
       month = {11},
       location = {Niagara Falls, ON, Canada},
       numpages = {7},
       doi = {10.1145/3009939.3009949},
       url = {http://dx.doi.org/10.1145/3009939.3009949},
       publisher = {ACM},
       keywords = {Spatial Interaction, 3D Visualization, Head-coupled Perspective, Information Visualization, Tangible Displays}
    }

    List of additional material

  • @article{fixme,
       author = {Wolfgang B\"{u}schel and Ricardo Langner and Ulrich von Zadow and Tom Horak and Raimund Dachselt},
       title = {Towards Cross-Surface Content Sharing Between Mobile Devices and Large Displays in the Wild},
       booktitle = {CHI 2016 Workshop on Interacting with Multi-Device Ecologies "'in the Wild"'},
       year = {2016},
       month = {5},
       location = {San Jose, CA, USA},
       numpages = {5}
    }

    List of additional material

Related Student Theses

  • Stefan Vogt

    Investigation of 3D Node Link Diagrams for the Visualization of Virtual Private Networks

    Stefan Vogt February 13th, 2017 until June 12th, 2017

    Supervision: Wolfgang Büschel, Raimund Dachselt