Documents in Your Hands: Exploring Interaction Techniques for Spatial Arrangement of Augmented Reality Documents
Abstract
Augmented Reality (AR) promises to enhance daily office activities involving numerous textual documents, slides, and spreadsheets by expanding workspaces and enabling more direct interaction. However, there is a lack of systematic understanding of how knowledge workers can manage multiple documents and organize, explore, and compare them in AR environments. Therefore, we conducted a user-centered design study (N = 21) using predefined spatial document layouts in AR to elicit interaction techniques, resulting in 790 observation notes. Thematic analysis identified various interaction methods for aggregating, distributing, transforming, inspecting, and navigating document collections. Based on these findings, we propose a design space and distill design implications for AR document arrangement systems, such as enabling body-anchored storage, facilitating layout spreading and compressing, and designing interactions for layout transformation. To demonstrate their usage, we developed a rapid prototyping system and exemplify three envisioned scenarios. With this, we aim to inspire the design of future immersive offices.
Media: Videos and Supplemental Material
5-min Video Figure
10-min Recorded Talk
Supplemental Material
Additionally, key components of our (6) data analysis are available. This includes an Excel sheet listing all identified and coded techniques from the user-centered studies, along with pre-processed and partially grouped notes from the post-study interviews.
Lastly, the rapid prototype system is openly available on GitHub. A detailed description of the system can be found in Section 7.1 of our paper.
Overview of Main Identified Interactions
Related Publication
@inproceedings{luo2025documents,
author = {Weizhou Luo and Mats Ole Ellenberg and Marc Satkowski and Raimund Dachselt},
title = {Documents in Your Hands: Exploring Interaction Techniques for Spatial Arrangement of Augmented Reality Documents},
booktitle = {Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems},
series = {CHI '25},
year = {2025},
month = {04},
isbn = {979-8-4007-1394-1/25/04},
location = {Yokohama, Japan},
numpages = {22},
doi = {10.1145/3706598.3713518},
url = {https://doi.org/10.1145/3706598.3713518},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
keywords = {spatial layout, content organization, interaction design, user-centered design, Mixed Reality}
}List of additional material
@inproceedings{luo2022arsensemaking,
author = {Weizhou Luo and Anke Lehmann and Hjalmar Widengren and Raimund Dachselt},
title = {Where Should We Put It? Layout and Placement Strategies of Documents in Augmented Reality for Collaborative Sensemaking},
booktitle = {Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems},
series = {CHI '22},
year = {2022},
month = {4},
location = {New Orleans, LA, USA},
numpages = {16},
doi = {10.1145/3491102.3501946},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {spatiality, spatial layout, content organization, sensemaking, affordance, qualitative user study, Augmented Reality, Mixed Reality, collaborative sensemaking}
}List of additional material
Acknowledgments
We thank Julie Hildebrandt for supporting the system implementation and Rufat Rzayev, Wolfgang Büschel, and Annett Mitschick for their help with the paper. This work was funded by the German Research Foundation (DFG, Deutsche Forschungsgemeinschaft) as part of Germany’s Excellence Strategy – EXC 2050/1 – Project ID 390696704 – Cluster of Excellence “Centre for Tactile Internet with Human-in-the-Loop” (CeTI) of Technische Universität Dresden and EXC-2068 – 390729961 – Cluster of Excellence “Physics of Life”, DFG grant 389792660 as part of TRR 248 (see https://perspicuouscomputing.science), the German Federal Ministry of Education and Research (BMBF, SCADS22B) and the Saxon State Ministry for Science, Culture and Tourism (SMWK) by funding the Center for Scalable Data Analytics and AI “ScaDS.AI Dresden/Leipzig“, and by the BMBF in the program of “Souverän. Digital. Vernetzt.”. Joint project 6G-life, project identification number: 16KISK001K.