Vision-Based Player Gesture Safety Control System
A raised-hand gesture component for tennis-machine stop control, first implemented separately and later consolidated into the shared person-attribute detector.
Gesture-based machine control
I first implemented raised-hand recognition as a separate perception component for the tennis machine. A player raising one or both hands provides the gesture used by the application’s stop-control logic.
The goal was to make the gesture available as a machine-readable signal while keeping it associated with the detected person.
Consolidation into person_attr
I later combined this capability with the person-attribute detector. Shared features now support person localization, depth estimation, raised-hand recognition, and player-role classification in a single model, removing the need for a separate learned gesture model in that path.
The integrated implementation predicts a hands-raised probability for each person. This is a binary attribute indicating that at least one hand is raised; it does not report a separate left/right hand class or a count of raised hands. Postprocessing keeps the probability attached to its person box, alongside depth and player-role output.
Labeling and evaluation
The later training-data workflow derives candidate hand-raise labels from SAM-3D-Body keypoints using 2D and 3D arm geometry. Its checks include wrist elevation, torso-relative direction, head clearance, elbow angle, and visibility, with uncertain cases available for review.
The repository also contains a comparison tool for learned hand-raise predictions and pose-derived gesture rules. It matches predicted people to labeled boxes before measuring gesture classification, helping distinguish a missed person from an incorrect gesture decision.
System boundary
The perception component supplies a person-associated gesture probability. Stop-command handling and physical actuation belong to the consuming controller, which interprets that signal together with the rest of the machine state.
