Graduation Year

2026

Document Type

Dissertation

Degree

Ph.D.

Degree Name

Doctor of Philosophy (Ph.D.)

Degree Granting Department

Computer Science and Engineering

Major Professor

Dmitry Goldgof, Ph.D.

Co-Major Professor

Yu Sun, Ph.D.

Committee Member

Peter R. Mouton, Ph.D.

Committee Member

Lawrence O. Hall, Ph.D.

Committee Member

Ashwin Parthasarathy, Ph.D.

Keywords

Computer Vision, Deep learning, Explainable AI, Interpretability, Machine Learning

Abstract

With the power of parallel processing, large datasets, and fast computational resources,deep neural networks (DNNs) have outperformed highly trained and experienced human experts in medical applications. However, the large global community of healthcare professionals, many of whom routinely face potentially life-or-death outcomes with complex medicolegal consequences, have yet to embrace this powerful technology. The major problem is that most current AI solutions function as a metaphorical black-box positioned between input data and output decisions without a rigorous explanation for their internal processes. With the goal of enhancing trust and improving acceptance of artificial intelligence– (AI) based technology in clinical medicine, there is a large and growing effort to address this challenge using eXplainable AI (XAI), a set of techniques, strategies, and algorithms with an explicit focus on explaining the “hows and whys” of DNNs. This dissertation focuses on developing XAI methods for neonatal pain assessment in black-box deep learning models.

First, we introduce data diagnosis for neonatal pain assessment using influence functions applied to neonatal audio spectrogram images. We employ influence functions to quantify the impact of individual training samples on model predictions. Experimental results demonstrate that the proposed approach can detect and remove harmful instances, leading to a compressed dataset. Furthermore, we extend the influence function by introducing a similarity-based weighting factor that incorporates representation-level proximity between training and test samples. The modified method improves the stability of influence estimation, leading to more reliable data-centric insights that also support subsequent model diagnosis.

Next, we introduce a concept-based model diagnosis framework that leverages a large Vision–Language Model (VLM) for automated concept discovery. Our method employs a VLM to select textual concepts that describe the classes in the target dataset. We then transform these influential textual concepts into human-readable image concepts using atext-to-image model. Further, we use directional derivatives to quantify the importance of the generated concepts. The results demonstrate that the VLM not only generates coherent and meaningful concepts that are easily understandable by non-experts but also achieves performance comparable to that of natural image concepts without the need for additional annotation costs.

Finally, we integrate data diagnosis with model diagnosis to establish a unified XAI framework that links predictive outcomes to both influential training instances and clinically meaningful concepts. This integrated approach provides deeper insight into model behavior, supports data-centric debugging, and facilitates clinically relevant interpretation of neonatal pain predictions. Overall, this dissertation contributes a novel XAI method that addresses the black-box nature of the model through modified influence functions and VLM-drivenconcept-based explanations that foster trust and promote adoption in clinical practice.

Download

Share

COinS