Document Type
Article
Publication Date
1-2021
Keywords
Confidence calibration, convolutional neural networks (CNNs), cost-sensitive learning, data imbalance, loss functions.
Digital Object Identifier (DOI)
https://doi.org/10.1109/TNNLS.2020.3047335
Abstract
Imbalanced class distribution is an inherent problem in many real-world classification tasks where the minority class is the class of interest. Many conventional statistical and machine learning classification algorithms are subject to frequency bias, and learning discriminating boundaries between the minority and majority classes could be challenging. To address the class distribution imbalance in deep learning, we propose a class rebalancing strategy based on a class-balanced dynamically weighted loss function where weights are assigned based on the class frequency and predicted probability of ground-truth class. The ability of dynamic weighting scheme to self-adapt its weights depending on the prediction scores allows the model to adjust for instances with varying levels of difficulty resulting in gradient updates driven by hard minority class samples. We further show that the proposed loss function is classification calibrated. Experiments conducted on highly imbalanced data across different applications of cyber intrusion detection (CICIDS2017 data set) and medical imaging (ISIC2019 data set) show robust generalization. Theoretical results supported by superior empirical performance provide justification for the validity of the proposed dynamically weighted balanced (DWB) loss function.
Rights Information
Was this content written or created while at USF?
Yes
Citation / Publisher Attribution
IEEE Transactions on Neural Networks and Learning Systems, in press
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Link to the publisher: https://doi.org/10.1109/TNNLS.2020.3047335
Scholar Commons Citation
Fernando, K. Ruwani M. and Tsokos, Chris P., "Dynamically Weighted Balanced Loss: Class Imbalanced Learning and Confidence Calibration of Deep Neural Networks" (2021). Mathematics and Statistics Faculty Publications. 29.
https://digitalcommons.usf.edu/mth_facpub/29
