Graduation Year

2020

Document Type

Dissertation

Degree

Ph.D.

Degree Name

Doctor of Philosophy (Ph.D.)

Degree Granting Department

Computer Science and Engineering

Major Professor

Sriram Chellappan, Ph.D.

Committee Member

Dmitry B. Goldgof, Ph.D.

Committee Member

Shaun Canavan, Ph.D.

Committee Member

Nasir Ghani, Ph.D.

Committee Member

Balaji Padmanabhan, Ph.D.

Keywords

CNN, Deep Learning, Mosquitoes, Public Health, Smart-phones

Abstract

According to WHO (World Health Organization) reports, among all animals, mosquitoes are responsible for the most deaths worldwide. Mosquito borne diseases continue to pose grave dangers to global health. In 2015 alone, 214 million cases of malaria were registered worldwide. According to Centers for Disease Control and Prevention (CDC) report published in 2016, 62,500 suspected case of Zika were reported to the Puerto Rico Department of Health (PRDH) out of which 29,345 cases were found positive. The year 2019 was recorded as the worst for dengue in South East Asia. There are close to 4,500 species of mosquitoes (spread across 34 or so genera, but only a select few are competent vectors. These vectors primarily belong to three genera - Aedes (Ae.), Anopheles ( An.) and Culex ( Cu.). Within these genera, there are multiple species responsible for transmitting particular diseases. Malaria is spread primarily by An. gambiae in Africa and by An. stephensi in India. Dengue, yellow fever, chikungunya, and the Zika fever are spread primarily by the species Ae. aegypti. Cu. nigripalpus is a vector for West Nile and other encephalitis viruses.

Since, not all types of mosquitoes spread diseases, in the case of any disease outbreak, an important first step is surveillance of vectors (i.e., those mosquitoes capable of spreading diseases). To do this today, public health workers lay several mosquito traps in the area of interest. Hundreds of mosquitoes will get trapped. Naturally, among these hundreds, taxonomists have to identify only the vectors to gauge their density. Unfortunately, species identification is still visual today, and is a laborious and very cognitively stressful process with trained personnel spending significant hours each day looking at each specimen with a microscope for accurate identification and recording.

In this dissertation, we first started by exploring the feasibility of developing an AI-enabled smart-phone based system to identify mosquito species using image based classification algorithm. We trained our algorithm on 303 images spread across 9 mosquito species that served more as a proof of concept to show that it is entirely feasible that common citizens also use our technique to identify species in their homes that can provide significant benefits to both residents and mosquito control programs in general. Our system integrates image processing, feature selection, unsupervised clustering, and an Support vector machine based machine learning algorithm for the species classification. The overall accuracy of our system for all 9 species is 77.5%.

After achieving encouraging results from our preliminary work, we collected more diverse mosquito images, which contains images taken with diverse array of smartphones, and in multiple backgrounds and orientations. With this larger scale dataset, we designed a deep learning architectures for three classes of problems a) to identify genus alone; b) to identify species based on knowledge of genus type; and finally c) to directly identify the species type. We also contrast the performance of each architecture and provide contextual relevance to ensuing results. Our CNN model based on Inception-ResNet V2 and Transfer Learning yielded an overall accuracy of 80% in classifying mosquitoes when trained on 25,867 images of 250 trapped mosquito vector specimens captured via many smart-phone cameras. In particular, the accuracy of our model in classifying Aedes aegypti and Anopheles stephensi mosquitoes (both of which are deadly vectors) are amongst the highest.

Next, to remove the effect of background noise as well as to concentrate the focus entirely on mosquito anatomy, we designed a framework based on state-of-the-art Mask Region-based Convolutional Neural Networks (Mask R-CNN) to automatically detect and separately extract anatomies of mosquitoes - thorax, wings, abdomen and legs from mosquito images. For this framework, we prepared a training dataset consists of 1500 smartphone images annotated with their mask anatomies across nine mosquito specimens trapped in Florida. In this framework, first, we classify objects of interest (foreground) from the background within the image. Then, we segment pixels containing anatomical components in the foreground by adding a branch to mask (i.e., extract pixels of) that component in the image, and in parallel we add two more branches to localize and classify the extracted anatomical components. The mAP for mask with 0.3, 0.5 and 0.7 IoUs are 0.625, 0.6, and 0.51 for validation dataset. The testing dataset mAP for mask with 0.3, 0.5 and 0.7 IoUs are 0.535, 0.524, and 0.412.

Further, we have done feasibility study of anatomy (thorax, wing, abdomen and leg) based classification for genus identification to improve the prediction accuracy for 3 genus category - Aedes, Anopheles and Culex. In this work, we conducted a feasibility study to identify these 3 mosquito genus from their smartphone images and anatomy-based deep neural network classification model. Very low intraclass variance among these mosquitoes genus and low quality images make this problem more challenging. To overcome this, we employed bilinear CNN architecture for our neural network model that works best in this scenario. we extracted four anatomies (thorax, abdomen, wings and legs) from each mosquito image and trained an independent model for each anatomy for genus classification. We also ensemble these models to compute the aggregated results. Our ensemble and 4 independent anatomy (thorax, abdomen, wing and leg) based model achieved 91%, 87.33%, 81%, 75.80% and 68.02% accuracies respectively on test data.

Download

Share

COinS