Abstract

The current practice of surgical telementoring involves remote experts providing guidance via telephone or video conferences. Especially in minimally invasive surgery (MIS), this greatly limits their insight into the procedure and inhibits effective deictic referencing. To address this, we present Endomersion, an immersive remote guidance and feedback system that allows verbal and non-verbal communication through audio chat, pointing and telestration, enables remote control of the endoscope camera, and offers an immersive 3D workspace supporting remote expert’s understanding of the on-site operation. The system was developed with feedback from surgeons and tested in actual operating room settings. In this demonstration, users take over the role of the remote expert, communicating with the on-site surgeon and remotely controlling the endoscopic camera attached to a robot arm.

Research Article


Download Preprint

Accompanying Video

https://www.youtube.com/watch?v=FIxNry4hf3M

Full video of Endomersion

Publications

  • Endomersion: An Immersive Remote Guidance and Feedback System for Robot-Assisted Minimally Invasive Surgery

    Endomersion: An Immersive Remote Guidance and Feedback System for Robot-Assisted Minimally Invasive Surgery

    Ellenberg, M.; Krug, K.; Fan, Y.; Krzywinski, J.; Dachselt, R.; Younis, R.; Wagner, M.; Weitz, J.; Rodriguez, A.; Just, G.; Bodenstedt, S.; Speidel, S.

    To appear in 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW).IEEE VR '25, Saint-Malo, France.IEEE,2025.

    @inproceedings{ellenberg2025endomersion,
       author = {Mats Ole Ellenberg and Katja Krug and Yichen Fan and Jens Krzywinski and Raimund Dachselt and Rayan Younis and Martin Wagner and J\"{u}rgen Weitz and Ariel Rodriguez and Gregor Just and Sebastian Bodenstedt and Stefanie Speidel},
       title = {Endomersion: An Immersive Remote Guidance and Feedback System for Robot-Assisted Minimally Invasive Surgery},
       booktitle = {2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)},
       series = {IEEE VR '25},
       year = {2025},
       month = {03},
       location = {Saint-Malo, France},
       numpages = {2},
       publisher = {IEEE},
       keywords = {Mixed Reality, Remote Guidance, Telestration, Minimally Invasive Surgery}
    }

    List of additional material

    Video

Acknowledgments

The authors acknowledge the financial support by the Federal Ministry of Education and Research of Germany in the programme of “Souverän. Digital. Vernetzt.”. Joint project 6G-life, project identification number: 16KISK001K.
We also acknowledge the support of the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy: EXC-2068, 390729961 – Cluster of Excellence “Physics of Life” and EXC 2050/1, 390696704 – Cluster of Excellence “Centre for Tactile Internet” (CeTI) of TU Dresden, and by DFG grant 389792660 as part of TRR 248 – CPEC (see https://cpec.science).