Graduation Year

2025

Document Type

Dissertation

Degree

Ph.D.

Degree Name

Doctor of Philosophy (Ph.D.)

Degree Granting Department

Electrical Engineering

Major Professor

Morris Chang, Ph.D.

Committee Member

Nasir Ghani, Ph.D.

Committee Member

Zhixin Miao, Ph.D.

Committee Member

Arman Sargolzaei, Ph.D.

Committee Member

Ehsan Sheybani, Ph.D.

Keywords

Deep Learning, Data-driven Methods, Embedded Deep Learning, Neural Architecture Search

Abstract

Deep neural networks have recently shown better performance than traditional machine learning algorithms in various applications such as computer vision, natural language processing, indoor navigation, and biomedical signal/bio-informatics data processing tasks. However, some main challenges still exist when harnessing these models for different applications. The first challenge is the performance of these methodologies for some specific types of data, such as Biomedical images, IMU measurements, and even bioinformatics data, such as genome expression. Moreover, deploying these models on mobile and IoT devices is computationally challenging. Most devices use cloud computing, where powerful deep-learning models analyze the data on a server. However, this approach can increase communication costs and render the system useless without communication.

To address these challenges, one approach is to develop a model with a meticulously designed architecture that shows reliable accuracy and efficiency in the target data. Different components and architectures can be considered to achieve reliable accuracy on specific types of data. Also, during the design, the efficiency of the model can be addressed so that it is computationally efficient, which means it can run on devices with limited processing power and memory.

If efficiency is the priority, the complexity of the model can be reduced using techniques such as quantization, pruning, and compression. Another approach is to use a hybrid framework combining cloud and on-device models. In this approach, the heavy computational load of the model is offloaded to a cloud server, while the low-end device handles only the input and output processing. This method allows more complex models to be used without overloading the device's resources. Additionally, this framework can dynamically decide which parts of the model should be executed on the cloud and which parts on the device, based on factors such as network connectivity, computational resources, and privacy concerns.

The thesis presents two distinct applications of deep learning methodology. The first chapter details the development of a low-end machine learning model using two innovative techniques: knowledge distillation and neural architecture search. The knowledge distillation method involves training a smaller and more computationally efficient model to learn from a larger and more complex model. In contrast, the neural architecture search method automates the design of neural network architectures to optimize performance on a specific task. Moreover, the first chapter proposes a new framework that efficiently utilizes server and client-side models. This framework is designed to overcome the limitations of low-end devices by offloading the heavy computational load to the server-side model. In contrast, the client-side model handles the input and output processing. The framework intelligently decides whether to operate on the local or the server model by extracting meta-information from each sample's classification result.

The second chapter presented in the thesis proposes a new deep-learning model to enhance the efficiency and effectiveness of indoor navigation tasks in mobile applications. The proposed model utilizes the advanced capabilities of deep learning to overcome the limitations of traditional navigation methods, which often rely on GPS and may not work effectively in indoor environments. Furthermore, a new dataset is introduced using the AR-Core API, which can help train and validate the proposed model. This dataset includes various indoor and outdoor spaces, such as offices, Hallways inside the buildings, and university campuses. It provides multiple scenarios to test the robustness and accuracy of the model. Overall, the second chapter in this thesis highlights the potential of deep learning methods to improve indoor navigation tasks in mobile applications, and the proposed dataset can aid in the development of more accurate and efficient navigation systems.

In the third chapter, the methodology outlined in the second chapter is refined and expanded to improve the model’s overall efficiency. This chapter delves into the integration of a Neural Architecture Search (NAS) framework, which systematically explores the architecture search space to discover the most effective neural network configurations. The objective is to balance accuracy and computational efficiency when applied to the target dataset.

A key contribution of this chapter is the introduction of a novel search space specifically designed for processing IMU measurements. Various candidate operators are proposed for different architecture components to optimize performance, ensuring that the resulting models are both efficient and well-suited for the task. Additionally, extensive experimental evaluations have been conducted to assess the effectiveness of the search method, analyzing different architecture configurations and their impact on accuracy, robustness, and computational cost. The results provide valuable insights into the advantages of using NAS for IMU-based machine learning tasks, highlighting its potential for real-world applications.

Download

Share

COinS