Graduation Year


Document Type




Degree Name

Doctor of Philosophy (Ph.D.)

Degree Granting Department

Computer Science and Engineering

Major Professor

Rangachar Kasturi, Ph.D.

Co-Major Professor

Dmitry Goldgof, Ph.D.

Committee Member

Yu Sun, Ph.D.

Committee Member

Richard Gitlin, Sc.D.

Committee Member

Terri Ashmeade, M.D.


Affective Computing, Computer Vision, Machine Learning, Medical Application, Video and Image Processing


For several decades, pediatricians used to believe that neonates do not feel pain. The American Academy of Pediatrics (AAP) recognized neonates' sense of pain in 1987. Since then, there have been many studies reporting a strong association between repeated pain exposure (under-treatment) and alterations in brain structure and function. This association has led to the increased use of anesthetic medications. However, recent studies found that the excessive use of analgesic medications (over-treatment) can cause many side effects. The current standard for assessing neonatal pain is discontinuous and suffers from inter-observer variations, which can lead to over- or under-treatment. Therefore, it is critical to address the shortcomings of the current standard and develop continuous and less subjective pain assessment tools.

This dissertation introduces an automatic and comprehensive neonatal pain assessment system. The presented system is different from the previous ones in three principal ways. First, it is specifically designed to assess pain of neonates using data captured while they are hospitalized in the Neonatal Intensive Care Units (NICU). Second, it dynamically analyzes neonatal pain as it unfolds in a particular pattern over time. Third, it combines visual, vocal, and physiological signals to create a system that continues to assess pain even when one or more signals become temporarily unavailable. The presented system has four main components. The first three components consist of novel algorithms for analyzing the visual, vocal, and physiological signals separately. The last component combines all the three signals to create a multimodal pain assessment system. The performance of the system in recognizing pain events is comparable to that of trained nurses; hence, it demonstrates the feasibility of automatic pain assessment in typical neonatal care environments.