Graduation Year
2024
Document Type
Dissertation
Degree
Ph.D.
Degree Name
Doctor of Philosophy (Ph.D.)
Degree Granting Department
Psychology
Major Professor
Brenton M. Wiernik, Ph.D.
Co-Major Professor
Stephen Stark, Ph.D.
Committee Member
Marina A. Bornovalova, Ph.D.
Committee Member
Georgia Chao, Ph.D.
Committee Member
John Licato, Ph.D.
Keywords
personality, derailers, text analysis, psychometrics, machine learning
Abstract
Advances in text analytic approaches to measuring personality traits from text and speech have shown great promise for automating the assessment of personality. While a lot of effort has gone into developing robust algorithms for predicting personality ratings, more work is needed in understanding the function and theoretical relationships of these algorithmic ratings. This study proposes a novel application of the Trait-Reputation-Identity model (McAbee and Connelly, 2016), wherein, algorithm-generated ratings form the Reputation factor and are assumed to resemble learned observer ratings, the original self-report ratings form the Identity factor, and the total set of ratings form the consensus Trait factor. To test this model, a set of open-ended text responses to assessment center exercises, along with self-reported personality scale scores will be modelled using two machine learning approaches: a) classification model – XGBoost, and b) deep learning model – RoBERTa. These models generated personality trait scores for a reserved set of text responses by “learning” from the input text. These scores, along with a combined Ensemble score, were compared to the original set of self-report scores for convergence, and were to assess criterion-relatedness on a set of assessment center competency scores. Several traits were shown to moderately replicate the expected direction of convergence with traits and criteria, however, several other traits failed to converge meaningfully. We provide a detailed discussion on the philosophy of thinking about algorithmic scores as potential “observers” with limitations addressed for future researchers to be mindful of when using machine learning for personality assessment. A great need for collaboration between psychometricians and computer scientists is uncovered and heavily advised.
Scholar Commons Citation
Raghavan, Mukhunth, "Are you who AI says you are? Exploring the role of Natural Language Processing algorithms for “predicting” personality traits from text" (2024). USF Tampa Graduate Theses and Dissertations.
https://digitalcommons.usf.edu/etd/10668
