Graduation Year
2022
Document Type
Thesis
Degree
M.S.C.S.
Degree Name
MS in Computer Science (M.S.C.S.)
Degree Granting Department
Computer Science and Engineering
Major Professor
Yu Sun, Ph.D.
Committee Member
Shaun Canavan, Ph.D.
Committee Member
Yasin Yilmaz, Ph.D.
Keywords
Continuous Action, Actor-Critic, DQN, Function Approximation, Robot Manipulation
Abstract
Pouring is one of the most commonly executed tasks in a variety of environments. Thereis less attention paid to pouring solid objects and avoiding spillage. Learning the dynamics for pouring solid objects can be a challenge because the collisions and static frictions between objects make their trajectories less predictable than liquid. Nonetheless, pouring solid objects is an important task in real life. In this work, we propose a solution to help robots aim and pour solid objects. The agents will learn how to interact with the environment and identify the optimal pouring trajectories, then manipulate the arm to aim at the target area by moving the end-effector in Inverse Kinematics (IK). The proposed solution is able to pour accurately into the target and minimize spillage. It also has the ability to generalize to unseen objects. Three reinforcement learning algorithms have been implemented, namely TD3, DDPG, and Double-DQN to address the proposed problem. Our study shows that the robot can learn to pour accurately by 1) using a continuous action space, and 2) avoiding overestimation in state-action value approximation.
Scholar Commons Citation
Li, Haoxuan, "Developing Reinforcement Learning Algorithms for Robots to Aim and Pour Solid Objects" (2022). USF Tampa Graduate Theses and Dissertations.
https://digitalcommons.usf.edu/etd/10318
