Toward Scalable ASL Education: Egocentric Stereo Sensing with LLM Feedback for Error-Aware Learning
An egocentric stereo learning system that detects how an ASL sign differs from an expert reference and turns those errors into clear, pedagogically grounded feedback.
The idea
Most ASL technology recognizes what someone signed. This work asks a different question: how can a learner improve the sign they just produced? The system compares a student’s motion with expert references and identifies parameter-level errors before generating feedback.
The technical core
Stereo cameras recover 3D hand motion in an egocentric frame. A multi-task model evaluates handshape, orientation, location, and movement, while confidence gating filters uncertain detections. Structured error records are then converted into feedback with a carefully constrained LLM prompt.
Why it matters
The approach creates a path toward scalable, on-demand practice while preserving the role of teachers and Deaf-community expertise in defining useful, trustworthy feedback.
