Journal Home
Login
Register
Open Access Journal
Submit a Paper
Propose a Special lssue
Open Access
ARTICLE
Immersive Technologies for Remote Human-Robot Interaction: A Systematic Literature Review
Issue Vol. 1 No. 01 (2024): Volume 01 Issue 01 --- Section Articles
pdf
Abstract
The integration of robotic systems into diverse applications necessitates robust and intuitive remote human-robot interaction (RHRI) capabilities. Traditional teleoperation methods often fall short in providing comprehensive situational awareness, immersive feedback, and intuitive control, leading to increased cognitive load and reduced performance. Extended Reality (XR), encompassing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), presents a promising paradigm shift for RHRI by offering enhanced immersion, rich sensory feedback, and natural interaction modalities. This systematic literature review comprehensively analyzes the current landscape of XR-enabled RHRI systems. We categorize existing approaches by XR technology (VR, AR, MR), examine various interaction modalities (visual, haptic, auditory, gesture, eye-tracking), and explore key application domains such as industrial manufacturing, hazardous environments, medical, and space exploration. The review highlights the significant benefits of XR in improving situational awareness, control intuition, and collaborative capabilities. Furthermore, it identifies persistent challenges, including latency, field of view limitations, usability, and hardware constraints. Finally, we discuss critical future research directions, emphasizing adaptive interfaces, standardization, enhanced multimodal feedback, and long-term human factors studies, to foster more efficient, safer, and natural human-robot interactions in remote settings.
Keywords
References
Mahmoud Abdulsalam and Nabil Aouf. 2023. VitRob pipeline: A seamless teleoperation pipeline for advanced virtual reality-robot interface applied for precision agriculture. In Proceedings of the 2023 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 1–6.
Dmytro Adamenko, Steffen Kunnen, Robin Pluhnau, André Loibl, and Arun Nagarajah. 2020. Review and comparison of the methods of designing the digital twin. Procedia CIRP 91 (2020), 27–32.
Zhuming Ai, Mark A. Livingston, and Ira S. Moskowitz. 2016. Real-time unmanned aerial vehicle 3D environment exploration in a mixed reality environment. In Proceedings of the 2016 International Conference on Unmanned Aircraft Systems (ICUAS). IEEE, 664–670.
Ian F. Akyildiz and Hongzhi Guo. 2022. Wireless communication research challenges for extended reality (XR). ITU Journal on Future and Evolving Technologies 3, 1 (2022), 1–15.
Rachel Albert, Anjul Patney, David Luebke, and Joohwan Kim. 2017. Latency requirements for foveated rendering in virtual reality. ACM Transactions on Applied Perception (TAP) 14, 4 (2017), 1–13.
Haider A.F. Almurib, Haidar Fadhil Al-Qrimli, and Nandha Kumar. 2012. A review of application industrial robotic design. In Proceedings of the 2011 9th International Conference on ICT and Knowledge Engineering. IEEE, 105–112.
Doris Aschenbrenner, Meng Li, Radoslaw Dukalski, Jouke Verlinden, and Stephan Lukosch. 2018. Collaborative production line planning with augmented fabrication. In Proceedings of the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 509–510.
Xue Bai, Changqiang Li, Keyan Chen, Yongjie Feng, Zhaowei Yu, and Ming Xu. 2018. Kinect-based hand tracking for first-person-perspective robotic arm teleoperation. In Proceedings of the 2018 IEEE International Conference on Information and Automation (ICIA). IEEE, 684–691.
Karlin Bark, Cuong Tran, Kikuo Fujimura, and Victor Ng-Thow-Hing. 2014. Personal navi: Benefits of an augmented reality navigational aid using a see-thru 3D volumetric HUD. In Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 1–8.
Zahraa Bassyouni and Imad H. Elhajj. 2021. Augmented reality meets artificial intelligence in robotics: A systematic review. Frontiers in Robotics and AI 8 (2021), 296.
Steve Benford, Chris Greenhalgh, Tom Rodden, and James Pycock. 2001. Collaborative virtual environments. Communications of the ACM 44, 7 (2001), 79–85.
Julio Betancourt, Baptiste Wojtkowski, Pedro Castillo, and Indira Thouvenin. 2022. Exocentric control scheme for robot applications: An immersive virtual reality approach. IEEE Transactions on Visualization and Computer Graphics 29, 7 (2022), 3392–3404.
Feifei Bian, Ruifeng Li, Lijun Zhao, Yihuan Liu, and Peidong Liang. 2018. Interface design of a human-robot interaction system for dual-manipulators teleoperation based on virtual reality. In Proceedings of the 2018 IEEE International Conference on Information and Automation (ICIA). IEEE, 1361–1366.
Frank Biocca, Arthur Tang, Charles Owen, and Fan Xiao. 2006. Attention funnel: Omnidirectional 3D cursor for mobile augmented reality platforms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1115–1122.
David Black, Yas Oloumi Yazdi, Amir Hossein Hadi Hosseinabadi, and Septimiu Salcudean. 2023. Human teleoperation-a haptically enabled mixed reality system for teleultrasound. Human–Computer Interaction 39, 5–6 (2023), 1–24.
Felix Bork, Christian Schnelzer, Ulrich Eck, and Nassir Navab. 2018. Towards efficient visual guidance in limited field-of-view head-mounted displays. IEEE Transactions on Visualization and Computer Graphics 24, 11 (2018), 2983–2992.
