Graduation Year
2025
Document Type
Thesis
Degree
M.S.
Degree Name
Master of Science (M.S.)
Degree Granting Department
Computer Science and Engineering
Major Professor
Zhao Han, Ph.D.
Committee Member
Jing Wang, Ph.D.
Committee Member
Julia Woodward, Ph.D.
Keywords
Human-robot interaction
Abstract
Entry to human-robot interaction research, e.g., conducting empirical experiments, faces a significant economic barrier due to the high cost of physical robots, ranging from thousands to tens of thousands. This cost issue also severely limits the field’s ability to replicate user studies and reproduce the results to verify their reliability, thus offering more confidence to incorporate these findings. Although virtual reality (VR) user studies present a potential solution, it is unclear whether we can confidently transfer the findings to physical robots and physical environments because VR isolates both the physical robot and the physical world where robots operate. To address this issue, we leveraged augmented reality (AR) only to simulate a virtual robot but retained the physical environment. Specifically, we implemented three conditions: (1) a physical robot interacted with a physical object, (2) a physical robot interacted with a virtual object, and (3) a virtual robot interacted with a virtual object. We used the Unity 3D engine and the Gazebo simulator to replicate the FetchIt! Mobile Manipulation Challenge environment and developed a communication bridge to synchronize the poses of virtual robots and objects with their physical counterparts in real-time. We implemented the pick-and-place operations for virtual and physical robots using the Robot Operating System (ROS), including designing custom ROS nodes for trajectory planning, motion execution, and arm-gripper publisher and subscriber between ROS and Unity. Moreover, we designed a human-subjects study, approved by the Institutional Review Board (IRB), to collect subjective experience measures, specifically usability, trust, and personal preference, and task performance across physical and virtual robot conditions. Although the user study will be conducted as part of future research, this thesis lays the technical groundwork by implementing all required robotic and AR components and validating their functionality across three experimental conditions.
Scholar Commons Citation
Kong, Xiangfei, "Bridging Virtual Robots and Physical Tasks via Augmented Reality" (2025). USF Tampa Graduate Theses and Dissertations.
https://digitalcommons.usf.edu/etd/10876
