Graduation Year

2024

Document Type

Dissertation

Degree

Ph.D.

Degree Name

Doctor of Philosophy (Ph.D.)

Degree Granting Department

Electrical Engineering

Major Professor

J. Morris Chang, Ph.D.

Committee Member

Ismail Uysal, Ph.D.

Committee Member

John Licato, Ph.D.

Committee Member

Lu Lu, Ph.D.

Committee Member

Zhuo Lu, Ph.D.

Keywords

Computer Vision, Natural Language Processing, Domain Generalization, Domain Adaptation

Abstract

Machine Learning (ML) and Deep Learning (DL) have achieved great success across diverse fields in the last decades, such as facial recognition, medical diagnosis, and language translation. The success, however, largely hinges on the assumption that the training and testing data share the same distribution or domain. In practice, real-world data often exhibits domain shifts, leading to notable degradation in model performance. Hence, the generalization capability of machine lea=rning models is of great importance, which refers to the domain generalization problem. In this dissertation, we present DADG, an effective algorithm for domain generalization, aiming to learn domain-invariant features from seen domains and boost the model classifier using cross-domain validation. In the experimental evaluation, a comprehensive comparison has been made among DADG and other 8 state-of-the-art algorithms. Extensive experiments have been conducted on 4 benchmark datasets. Our results show that DADG outperforms the other algorithms on 3 datasets.

As a universal problem across different areas in deep learning, domain shift is also exhibited in Natural Language Processing. We propose Epi-Curriculum, an effective learning algorithm for low-resource domain adaptation in Neural Machine Translation (NMT). It effectively improves the model's robustness to domain shift and adaptability when only hundreds of parallel training data are available in the target domain. Extensive experiments have been conducted on 3 different translation tasks. Our results show that Epi-Curriculum outperforms the other 5 methods in terms of the BLEU score.

One key principle of our DADG and Epi-Curriculum is to learn domain-invariant features across different seen domains, which refers to representation learning. Representation learning is foundational to the success of deep learning, varying widely in its forms depending on the specific learning objectives. Such approaches not only overcome domain-specific challenges but also provide new solutions for related issues inherent in machine learning tasks. Among these, class imbalance stands out as a critical challenge, characterized by a disproportionate distribution of classes that adversely affects model accuracy. This imbalance often results in models that are biased toward the majority class, overlooking minority classes due to the uniform weight attributed to each sample during training. Traditional solutions have predominantly focused on adjusting the sampling ratio or modifying loss functions, approaches that do not address the fundamental objective of deep learning: the extraction of meaningful features. We propose SuperCon, an effective learning framework, aiming to learn distinguishable features for each class using supervised contrastive loss. In the experimental evaluation, a comprehensive comparison has been made between SuperCon and the other 5 conventional algorithms. Our results show that SuperCon significantly outperforms the conventional methods and effectively extracts differentiated features.

As an effective method for integrating multiple single models, ensemble learning significantly enhances the performance beyond that achievable by any individual model. However, a primary challenge lies in effectively adjusting the strengths and weaknesses of each model to optimize their contributions to the final prediction. We present CS-AF, an ensemble learning algorithm that actively determines the influence of each single model with an application-sensitive cost matrix. In the experimental evaluation, CS-AF outperforms 4 other state-of-the-art algorithms in terms of accuracy and pre-defined cost on a benchmark skin lesion image dataset, showcasing the effectiveness of CA-AF.

Download

Share

COinS