Graduation Year
2024
Document Type
Dissertation
Degree
Ph.D.
Degree Name
Doctor of Philosophy (Ph.D.)
Degree Granting Department
Computer Science and Engineering
Major Professor
Yu Sun, Ph.D.
Committee Member
John Licato, Ph.D.
Committee Member
Gene Louis Kim, Ph.D.
Committee Member
Mingyang Li, Ph.D.
Committee Member
Seden Dogan, Ph.D.
Keywords
Robotics, Knowledge Graph, Foundation Model, Large Language Model, Error Recovery
Abstract
The deployment of robotic systems across various domains has expanded significantly, with applications ranging from domestic tasks, such as cleaning and cooking, to industrial operations requiring precision and automation. These advancements highlight the critical importance of effective task planning in ensuring that robots can perform tasks safely, efficiently, and autonomously. However, task planning in robotics faces challenges related to generalization, executability, flexibility, and limitations in existing knowledge bases.
This dissertation addresses these challenges through innovative strategies aimed at enhancing robotic task planning and execution. We first focus on adapting to unknown scenarios by utilizing the Functional Object-Oriented Network (FOON) to learn how to manipulate novel objects through semantic similarity. This improves adaptability in task planning for robotic cooking.
The work also explores leveraging large language models (LLMs) to achieve generalization in task planning. Recognizing the tendency of LLMs to produce unreliable outputs, our approach focuses on improving the reliability of these plans, using Planning Domain Definition Language (PDDL) to enhance flexibility and robustness.
Furthermore, we introduce a strategy that integrates foundation models with knowledge graphs to address automatic failure recovery, reducing dependency on LLMs and minimizing hallucination issues. We enhance task planning accuracy and system resilience by introducing an effective method for failure detection and recovery. To validate this approach, we created a specialized dataset that includes a variety of failure scenarios, facilitating comprehensive evaluation of our strategies.
The outcomes of this research present a robust framework for developing intelligent robotic systems capable of adaptive, efficient, and autonomous operations, thus paving the way for the next generation of robots to seamlessly integrate and enhance diverse human activities.
Scholar Commons Citation
Sakib, Md Sadman, "Developing Robotic Task Planning Methods for Diverse Real-World Challenges" (2024). USF Tampa Graduate Theses and Dissertations.
https://digitalcommons.usf.edu/etd/10675
