Start Date
1-5-2026 12:00 PM
End Date
1-5-2026 1:00 PM
Description
As technology becomes increasingly integrated into daily life, seamless and multimodal human-robot interaction has become essential. This project presents a lightweight, real-time human-following system implemented on a differential-drive robot (Ham-Bot) using a Raspberry Pi, motivated by the growing need for robotic companionship among elderly and mobility-impaired individuals. The system integrates HSV-based color segmentation, MediaPipe hand gesture recognition, LiDAR-based distance regulation, and a six-state finite state machine to achieve robust human-robot interaction without relying on heavy machine learning models for body detection. A socket-based communication architecture streams camera frames from the robot to a MacOS client, where perception is performed and control commands are returned in real time. A REDETECT state enables the robot to autonomously recover from target loss by spinning in the last known direction until the color signal is reacquired. A custom hand gesture dataset was collected using MediaPipe keypoint classification and used to train a TensorFlow Lite model, supporting five gesture classes, with dual PID controllers regulating lateral alignment and longitudinal distance independently. Experimental evaluation demonstrates tracking success rates of 92.6%, mean distance error of 383 mm from the 500 mm target, and state transition accuracy of 98.5%. The findings confirm that robust human following is achievable on resource-constrained hardware through multimodal sensor fusion and autonomous recovery strategies.
Hand Gesture Control for a Di2erential Drive Robot
As technology becomes increasingly integrated into daily life, seamless and multimodal human-robot interaction has become essential. This project presents a lightweight, real-time human-following system implemented on a differential-drive robot (Ham-Bot) using a Raspberry Pi, motivated by the growing need for robotic companionship among elderly and mobility-impaired individuals. The system integrates HSV-based color segmentation, MediaPipe hand gesture recognition, LiDAR-based distance regulation, and a six-state finite state machine to achieve robust human-robot interaction without relying on heavy machine learning models for body detection. A socket-based communication architecture streams camera frames from the robot to a MacOS client, where perception is performed and control commands are returned in real time. A REDETECT state enables the robot to autonomously recover from target loss by spinning in the last known direction until the color signal is reacquired. A custom hand gesture dataset was collected using MediaPipe keypoint classification and used to train a TensorFlow Lite model, supporting five gesture classes, with dual PID controllers regulating lateral alignment and longitudinal distance independently. Experimental evaluation demonstrates tracking success rates of 92.6%, mean distance error of 383 mm from the 500 mm target, and state transition accuracy of 98.5%. The findings confirm that robust human following is achievable on resource-constrained hardware through multimodal sensor fusion and autonomous recovery strategies.