Virtual Reality Control of a Planetary Rover Manipulator Using Unity and Unreal Engine

Authors

  • Paweł Kwiatoń Department of Mechanics and Machine Design Fundamentals, Faculty of Mechanical Engineering, Czestochowa University of Technology, Dąbrowskiego 73, 42-201 Częstochowa, Poland
  • Krzysztof Sokół Department of Mechanics and Machine Design Fundamentals, Faculty of Mechanical Engineering, Czestochowa University of Technology, Dąbrowskiego 73, 42-201 Częstochowa, Poland
  • Grzegorz Ptak Department of Mechanics and Machine Design Fundamentals, Faculty of Mechanical Engineering, Czestochowa University of Technology, Dąbrowskiego 73, 42-201 Częstochowa, Poland
  • Piotr Ptak Department of Mechanics and Machine Design Fundamentals, Faculty of Mechanical Engineering, Czestochowa University of Technology, Dąbrowskiego 73, 42-201 Częstochowa, Poland
  • Dawid Cekus Department of Mechanics and Machine Design Fundamentals, Faculty of Mechanical Engineering, Czestochowa University of Technology, Dąbrowskiego 73, 42-201 Częstochowa, Poland

Keywords:

virtual reality, control system, planetary rover, manipulator, game engines

Abstract

This paper presents a method for controlling a manipulator mounted on a planetary rover using virtual reality (VR) technology. The proposed approach combines VR controllers with stereovision cameras to provide the operator with an improved view of the manipulator workspace. Unity and Unreal Engine were used to visualize the rover and the manipulator in a virtual environment. The proposed control system was implemented and tested on a real rover, and its operation was compared with a conventional joystick-based control system. In addition, the performance of both graphics engines was evaluated during object rendering operations using selected CPU, GPU, RAM, and frame-rate metrics. The results indicate that the proposed VR-based approach can reduce task completion time while providing immersive visual feedback during manipulator operation. The developed system can also be adapted to other mobile or industrial platforms equipped with manipulators.

References

[1] H. Yang, Application technology of space manipulator, In Manned Spacecraft Technologies, Springer Singapore, 2021, 295–309.

[2] L. G. Teoh, Y.-J. Tseng, and C.-S. Chen, Applying virtual reality to the control system for status judgment and arrangement of workpiece, 2019 IEEE International Conference on Computation, Communication and Engineering (ICCCE), Fujian, China, 2019, 40–43.

[3] R. Li, M. van Almkerk, S. van Waveren, E. J. Carter, and I. Leite, Comparing human-robot proxemics between virtual reality and the real world, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Daegu, South Korea, 2019, 431–439.

[4] L. Gammieri, M. Schumann, L. Pelliccia, G. D. Gironimo, and P. Klimant, Coupling of a redundant manipulator with a virtual reality environment to enhance human-robot cooperation, Procedia CIRP, 62, 2017, 618–623.

[5] P. Ptak, K. Sokoł, and D. Cekus, Design of the manipulator control unit for modernity mars rover, Engineering Mechanics 2019, Prague, 2019, 299–302.

[6] N. Sreekanth, A. Dinesan, A. R. Nair, G. Udupa, and V. Tirumaladass, Design of robotic manipulator for space applications, Materials Today: Proceedings, 46, 2021, 4962–4970.

[7] M. Pierzgalski, P. Ptak, D. Cekus, and K. Sokoł, Modeling and stress analysis of a manipulator mounted on a mars rover, Procedia Engineering, 177, 2017, 121–126.

[8] D. Skrobek and D. Cekus, Optimization of the operation of the anthropomorphic manipulator in a three-dimensional working space, Engineering Optimization, 51, 2019, 1997–2010.

[9] T. Szcz˛esny and M. Re´cko, Control of robotic arm for mars rover analogue, 2018 19th International Carpathian Control Conference (ICCC), Szilvasvarad, Hungary, 2018, 259–264.

[10] A. P. Buczyłowski, M. Re´cko, A. Wolniakowski, and K. Haraburda, A hand-held teleoperation device for the control of the mars rover analogue manipulator, International Journal of Mechanical Engineering and Robotics Research, 9, 2020, 1018–1023.

[11] Kubik and P. Kwiato´n, Forward kinematics algorithm for anthropomorphic manipulators, Prace Naukowe Akademii im. Jana Długosza w Cz˛estochowie. Technika, Informatyka, In˙zynieria Bezpiecze´nstwa, 5, 2017, 39–51.

[12] P. Lu, Z.-W. Ren, and K.-T. Zhang, Control system design and kinematic analysis for a 4-dof light weight manipulator, 2017 Chinese Automation Congress (CAC), Jinan, China, 2017, 5673–5678.

[13] F. Ropero, P. Munoz, M. D. R-Moreno, and D. F. Barrero, A virtual reality mission planner for mars rovers, 2017 6-th nternational Conference on Space Mission Challenges for Information Technology (SMC-IT), California, USA, 2017, 142–146.

[14] M. F. Alam, S. Katsikas, O. Beltramello, and S. Hadjiefthymiades, Augmented and virtual reality based monitoring and safety system: A prototype IoT platform, Journal of Network and Computer Applications, 89, 2017, 109–119.

[15] N. Zhang, T. Qi, and Y. Zhao, Real-time learning and recognition of assembly activities based on virtual reality demonstration, Sensors, 21, 2021, 6201.

[16] G. Garg, V. Kuts, and G. Anbarjafari, Digital twin for FANUC robots: Industrial robot programming and simulation using virtual reality, Sustainability, 13, 2021, 10336.

[17] L. Perez, S. Rodriguez-Jimenez, N. Rodriguez, R. Usamentiaga, and D. F. Garcia, Digital twin and virtual reality based methodology for multi-robot manufacturing cell commissioning, Applied Sciences, 10, 2020, 3633.

[18] O. Liu, D. Rakita, B. Mutlu, and M. Gleicher, Understanding human-robot interaction in virtual reality, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Lisbon, Portugal, 2017, 751–757.

[19] A. A. Malik, T. Masood, and A. Bilberg, Virtual reality in manufacturing: Immersive and collaborative artificial-reality in design of human-robot workspace, International Journal of Computer Integrated Manufacturing, 33, 2020, 22–37.

[20] X. Sha, Z. Jia, W. Sun, Y. Hao, X. Xiao, and H. Hu, Development of mixed reality robot control system based on HoloLens, In Intelligent Robotics and Applications, Springer International Publishing, 2019, 571–581.

[21] B. Bejczy, R. Bozyil, E. Vaiˇcekauskas, S. B. K. Petersen, S. Bogh, S. S. Hjorth, and E. B. Hansen, Mixed reality interface for improving mobile manipulator teleoperation in contamination critical applications, Procedia Manufacturing, 51, 2020, 620–626.

[22] D. Cafolla, M. Russo, and M. Ceccarelli, Experimental validation of HeritageBot III, a robotic platform for cultural heritage, Journal of Intelligent and Robotic Systems, 100, 2020, 223–237.

[23] T. Namba and Y. Yamada, Risks of deep reinforcement learning applied to fall prevention assist by autonomous mobile robots in the hospital, Big Data and Cognitive Computing, 2, 2018, 13.

[24] A. Khadr and A. Houidi, Designing digital twin of a three wheeled robot using simscape for trajectory tracking, Applications of Modelling and Simulation, 10, 2026, 163–172.

[25] Alkamachi, Adaptive pd-based control of robot manipulators with gravity compensation and rbf neural networks, Applications of Modelling and Simulation, 10, 2026, 108–119.

[26] R. Galarza, P. Ayala, S. Manzano, and M. V. Garcia, Virtual reality teleoperation system for mobile robot manipulation, Robotics, 12, 2023, 163.

[27] M. Gallipoli, S. Buonocore, M. Selvaggio, G. A. Fontanelli, S. Grazioso, and G. D. Gironimo, A virtual reality-based dual-mode robot teleoperation architecture, Robotica, 42, 2024, 1935–1958.

[28] A. George, A. Bartsch, and A. B. Farimani, OpenVR: Teleoperation for manipulation, SoftwareX, 29, 2025, 102054.

[29] Skrobek, Modeling, Analysis and Optimization of the Work Cycle of Manipulators with Four Degrees of Freedom, Ph.D. Thesis, Czestochowa University of Technology, 2021.

[30] G. Ptak, The Use of Virtual Reality to Control the Mars Rover’s Manipulator, M.Sc. Thesis, Czestochowa University of Technology, 2021.

[31] Blender. https://www.blender.org/, (accessed 28 august 2026).

[32] Unity. https://unity.com/, (accessed 28 august 2026).

[33] OpenCV documentation. https://docs.opencv.org/, (accessed 28 august 2026).

[34] Unreal engine. https://www.unrealengine.com/, (accessed 28 august 2026).

[35] Y. Sahraoui, A. Ghanam, S. Zaidi, S. Bitam, and A. Mellouk, Performance evaluation of TCP and UDP based video streaming in vehicular ad-hoc networks, 2018 International Conference on Smart Communications in Network Technologies (SaCoNeT), El Qued, Algeria, 2018, 67–72.

[36] Z. Zhong and W. Hu, Error detection and control of IIoT network based on CRC algorithm, Computer Communications, 153, 2020, 390–396.

[37] A. Ciekanowska, A. K. Gli´nski, and K. Dziedzic, Comparative analysis of Unity and Unreal Engine efficiency in creating virtual exhibitions of 3d scanned models, Journal of Computer Sciences Institute, 20, 2021, 247–253.

[38] Christopoulou and S. Xinogalos, Overview and comparative analysis of game engines for desktop and mobile devices, International Journal of Serious Games, 4, 2017, 21–36.

Downloads

Published

28-10-2026

How to Cite

Kwiatoń, P., Sokół, K., Ptak, G., Ptak, P., & Cekus, D. (2026). Virtual Reality Control of a Planetary Rover Manipulator Using Unity and Unreal Engine. Applications of Modelling and Simulation, 10, 291–302. Retrieved from http://arqiipubl.com/ojs/index.php/AMS_Journal/article/view/1523

Issue

Vol. 10 (2026)

Section

Articles

License

Copyright (c) 2026 Paweł Kwiatoń, Krzysztof Sokół, Grzegorz Ptak, Piotr Ptak, Dawid Cekus

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with this journal agree to the following terms:

Authors hold and retain copyright, and grant the journal right of first publication, with the work after publication simultaneously licensed under a Creative Commons Attribution 4.0 License CC BY that permits any use, reproduction and distribution of the work and article without further permission provided that the original work is properly cited.

Authors are permitted and encouraged to post their work online in institutional repositories, website and other social media before and after publication, as it can lead to productive exchanges, as well as earlier and greater citation of published work.

Similar Articles

<< < 1 2 3 4 5 6 7 > >> 

You may also start an advanced similarity search for this article.