Evaluating Conversation Agent Impact on Student Experience in a Distance Education Course

Author

Grover Walters, University of South Florida

Graduation Year

2019

Document Type

Dissertation

Degree

D.B.A.

Degree Granting Department

Information Systems and Decision Sciences

Major Professor

Joann Quinn, Ph.D.

Co-Major Professor

Sunil Mithas, Ph.D.

Committee Member

Moez Limayem, Ph.D.

Committee Member

Steven Oscher, D.B.A.

Keywords

Amazon Alexa Skill, Conversational Learning Object Scale, Online Learning Experience, Virtual Assistant

Abstract

We explore the efficacy of conversation agents operating as an instructional aid in a distance education course. Two aspects of efficacy are considered—conversation agent impact on student perceptions of the experience, and how different design features of the agent affect student perceptions of engagement. Evaluation of the agent is accomplished by collecting data from 24 undergraduate participants separated into random groups. We conduct two rounds of mixedmethod evaluation. Between the two rounds, a modification to the agent occurs based on the outcome of the first evaluation. Findings include limitations related to phrasing and data persistence features of the design that initially yielded less-than-favorable perceptions of the agent prototype.

Scholar Commons Citation

Walters, Grover, "Evaluating Conversation Agent Impact on Student Experience in a Distance Education Course" (2019). USF Tampa Graduate Theses and Dissertations.
https://digitalcommons.usf.edu/etd/8088