Graduation Year

2019

Document Type

Thesis

Degree

M.S.

Degree Name

Master of Science (M.S.)

Degree Granting Department

Computer Science and Engineering

Major Professor

Shaun Canavan, Ph.D.

Committee Member

Sudeep Sarkar, Ph.D.

Committee Member

Tempestt Neal, Ph.D.

Keywords

machine learning, deepfake, deep learning, computer vision, convolutional neural network

Abstract

We use Rossler’s FaceForensics dataset of 1004 online videos and their corresponding forged counterparts [1] to investigate the ability to distinguish digitally forged facial images from original images automatically with deep learning. The proposed convolutional neural network is much smaller than the current state-of-the-art solutions. Nevertheless, the network maintains a high level of accuracy (99.6%), all while using the entire FaceForensics dataset and not including any temporal information. We implement majority voting and show the impact on accuracy (99.67%), where only 1 video of 300 is misclassified. We examine why the model misclassified this one video. In terms of tuning the network, we observe how changing hyperparameters affects training time for each epoch and accuracy for training, validation, and testing datasets. There are some challenges involved in obtaining consistent results with deep learning because of the randomization involved in initializing weights. We also replicate Rossler’s XceptionNet [2] experiment for classifying images as originals or forgeries and examine the underlying issues with his research: using a subset of data that is not representative of the full dataset and lack of generalization because of network overfitting when using transfer learning with XceptionNet. Lastly, we explore future work, including forged audio, different network types, and new image datasets.

Download

Share

COinS