Master of Science (M.S.)
Degree Granting Department
Computer Science and Engineering
Shaun Canavan, Ph.D.
Sudeep Sarkar, Ph.D.
Tempestt Neal, Ph.D.
machine learning, deepfake, deep learning, computer vision, convolutional neural network
We use Rossler’s FaceForensics dataset of 1004 online videos and their corresponding forged counterparts  to investigate the ability to distinguish digitally forged facial images from original images automatically with deep learning. The proposed convolutional neural network is much smaller than the current state-of-the-art solutions. Nevertheless, the network maintains a high level of accuracy (99.6%), all while using the entire FaceForensics dataset and not including any temporal information. We implement majority voting and show the impact on accuracy (99.67%), where only 1 video of 300 is misclassified. We examine why the model misclassified this one video. In terms of tuning the network, we observe how changing hyperparameters affects training time for each epoch and accuracy for training, validation, and testing datasets. There are some challenges involved in obtaining consistent results with deep learning because of the randomization involved in initializing weights. We also replicate Rossler’s XceptionNet  experiment for classifying images as originals or forgeries and examine the underlying issues with his research: using a subset of data that is not representative of the full dataset and lack of generalization because of network overfitting when using transfer learning with XceptionNet. Lastly, we explore future work, including forged audio, different network types, and new image datasets.
Scholar Commons Citation
Sambhu, Neilesh, "Detecting Digitally Forged Faces in Online Videos" (2019). USF Tampa Graduate Theses and Dissertations.