Transforming Organizational Cyber Security with Artificial Intelligence and Data-driven Optimization

Graduation Year

2024

Document Type

Dissertation

Degree

Ph.D.

Degree Name

Doctor of Philosophy (Ph.D.)

Degree Granting Department

Industrial and Management Systems Engineering

Major Professor

Ankit Shah, Ph.D.

Committee Member

Tapas K. Das, Ph.D.

Committee Member

Xinming Ou, Ph.D.

Committee Member

Hadi Gard, Ph.D.

Committee Member

Balaji Padmanabhan, Ph.D.

Keywords

Deep learning, Deep reinforcement learning, Evasion attacks, Network intrusion detection systems, Optimization under uncertainty, Vulnerability management

Abstract

This dissertation presents a comprehensive framework for enhancing organizational cybersecurity through data-driven intelligence. The research integrates multiple methodologies to tackle challenges in network intrusion detection and vulnerability management within cybersecurity operations centers (CSOCs). First, the research investigates vulnerability prioritization and mitigation techniques currently employed by CSOCs. To further streamline the vulnerability prioritization and mitigation process a machine learning (ML)-based Vulnerability Priority Scoring System (VPSS) is introduced, significantly improving the prioritization and mitigation of context-sensitive vulnerabilities. The VPSS outperforms traditional methods, reducing the cumulative vulnerability exposure score by up to 30% by considering both organizational context and vulnerability severity. Next, the research develops Deep PackGen, a Deep Reinforcement Learning (DRL) framework that performs red team evaluation of anomaly-based Network Intrusion Detection Systems (NIDS). The DRL-based methodology evades ML-based NIDS, achieving a 66.4% average adversarial success rate across various ML models, thereby demonstrating the shortcomings of current intrusion detection mechanisms. Finally, the research develops a robust NIDS framework, Deep ResNIDS that incorporates malicious packet detection, anomaly detection, and novelty detection, demonstrating robust performance with a 99% accuracy rate in identifying malicious packets. Furthermore, the study evaluates autoencoder models for anomaly detection in packet-based NIDS, emphasizing the efficiency of advanced deep learning techniques in improving detection capabilities. Overall, this work highlights the effectiveness of advanced analytics and optimization frameworks in enhancing cybersecurity posture, optimizing resource allocation, and providing scalable, customizable solutions tailored to specific organizational needs.

This document is currently not available here.

Share

COinS