Graduation Year

2021

Document Type

Dissertation

Degree

Ph.D.

Degree Name

Doctor of Philosophy (Ph.D.)

Degree Granting Department

Curriculum and Instruction

Major Professor

Sanghoon Park, Ph.D.

Committee Member

Tom Miller, Ph.D.

Committee Member

David Lamb, Ph.D.

Committee Member

James Hatten, Ph.D.

Keywords

Academic Advising, Chatbots, Human Performance Technology, Technology Acceptance Model (TAM)

Abstract

Academic advising is seen increasingly as an essential function supporting the educational mission of universities. Institutions have made significant investments supporting academic advisors and student success initiatives by leveraging technology resources such as Canvas and Degreeworks. There is a rising need for technology that can assist academic advisors in their responsibility to guide students along the path to academic success and AI resources that promote more efficient use of human capital are positioned to meet that need. Chatbot applications can serve as a technology-mediated intervention for delivering information and interactive content to support student success.

This study aimed to design, develop, and evaluate an automated, AI-enabled advising resource (the AVA chatbot) to deliver academic advising and university-related information in an honors college environment. Human Performance Technology (HPT) served as a framework throughout the study to shape the conversation around performance gaps, advising information delivery, and using chatbot technology as an intervention. During the organizational analysis, the HPT model provided a workflow for understanding the honors college values, mission, and vision. The study indicated that the AVA chatbot added value in the sharing of real-time data with advisors and that it was perceived as having acceptable response accuracy by participants.

The real-time data visible to stakeholders and decision-makers offered a better understanding of students’ needs and was a benefit to students' wellbeing. The AVA data was successful at showing where information gaps existed and when students were asking questions relevant to their academic journey. Technology Assessment Model (TAM) survey data yielded valuable insight into who used the chatbot and how participants engaged with it. Extracted AVA data contributed to addressing needs that were not met via university websites. The data also showed which student populations felt more supported by the AVA through their successful interaction with the chatbot. The AVA chatbot data illustrated opportunities for improvement to stakeholders and showed paths to addressing performance gaps. Additionally, the AVA data provided helpful insights about participants which helped to enhance the AVA chatbot and student portal site.

Finally, the researcher conducted a comprehensive evaluation to assess the appropriateness of the AVA's responses to participant questions. Positive perceptions toward the AVA chatbot were evidenced by the data on perceived ease of use, perceived usefulness, and intention to use. Of these three survey categories, the AVA scored lowest on the intention to use and highest on perceived ease of use, with the majority of participants indicating they would use the AVA again.

The findings concluded that chatbots could help large and small higher education institutions effectively support their student populations by providing round-the-clock advising information as well as tracking and measuring overall contributions and pinpointing data insights in real-time. The implications of this study can be valuable to both technology implementation in higher education and chat application design and development.

Download

Share

COinS