SignGlass: First-Person View Comprehensive and Generalizable ASL Translation Using Wearable Glass
Wearable smart glasses that capture manual and non-manual American Sign Language markers, then translate continuous signing into natural English.
The idea
SignGlass moves ASL translation into a first-person, wearable form factor. Three cameras jointly observe hand motion, body context, and facial expressions—the manual and non-manual signals that carry meaning in ASL.
The technical core
The system combines jitter-aware spatio-temporal attention for hand movement, a temporally aware facial module, and cascaded augmentation for stronger generalization across different signing styles. The resulting pipeline supports continuous recognition and translation rather than isolated-sign classification.
Why it matters
By bringing comprehensive sensing into a familiar pair of glasses, the work points toward more portable and inclusive communication tools for Deaf and Hard-of-Hearing communities.
