evidoria

← Back to browse

Good practice Imported

NAO Robot Colombian Sign Language Tutor — Universidad de La Sabana's LSTM-Powered Vocabulary Trainer

Colombia · Chía · See the Colombia profile

Evidence: Descriptive / self-reported Top 89% 27/100 · Ask Evidence Copilot about this practice

A team from Universidad de La Sabana taught a NAO humanoid robot to teach and check basic Colombian Sign Language vocabulary, hitting 91.6-93.8% sign-recognition accuracy in lab tests — a promising prototype the authors admit has not yet been tried with deaf students.

91.6 %
Sign-recognition F1 score (webcam) (lab testing)
93.8 %
Sign-recognition F1 score (NAO camera) (lab testing)
85 milliseconds
Response time (webcam) (lab testing)
2.3 seconds
Response time (NAO camera) (lab testing)
20 signers
Signers used to train the model (n/a)
NAO Robot Colombian Sign Language Tutor — Universidad de La Sabana's LSTM-Powered Vocabulary Trainer

Details

Maturity
Pilot
Promoter
Universidad de La Sabana (CAPSAB Research Group, Faculty of Engineering)
Period
2023–2024
Keywords
higher education, robotics, sign language, deaf education

Context

A team from Universidad de La Sabana's CAPSAB control-and-automation research group in Chia, Colombia, programmed a NAO H25 humanoid robot to teach and check basic Colombian Sign Language (CSL) vocabulary, starting with 11 colour signs, in response to a shortage of qualified CSL teachers - fewer than 0.2% of Colombian educational institutions offer bilingual instruction for deaf pupils.

Objectives

Build and technically validate an AI-driven sign-language tutoring system before any classroom or learner testing.

Activities

The system extracts hand and torso coordinates from video using MediaPipe and classifies signs with a recurrent (LSTM) neural network trained on 20 signers, tested via either a webcam or the robot's own camera communicating over TCP/IP.

Results

In technical testing, the system reached an F1 score of 91.6% (webcam) and 93.8% (NAO camera), with response times of 85 milliseconds and 2.3 seconds respectively; the sign for 'purple' was recognised with 100% accuracy.

Conclusions

This is early-stage, technology-readiness evidence rather than classroom evidence: the paper reports sign-recognition performance only, and the authors state plainly that the system has not yet been tested with deaf learners - an educational-effectiveness trial is still needed before any claim of learning impact would be warranted.

Implementation
Indicative cost
Low (< €50k) — No cost figure stated; low band is a conservative inferred estimate for a university research prototype built on an existing NAO robot and open-source tools.
Time to results
Short (< 1 year) — Research and testing phase reported for 2023-2024.
Staffing & skills
Universidad de La Sabana CAPSAB research group, Faculty of Engineering

Conditions for success

  • Built on established open tools (MediaPipe for pose extraction) and a purpose-trained LSTM network using data from 20 real signers

Common failure modes

  • System has not been tested with deaf learners at all, only technically validated
  • Vocabulary limited to 11 colour signs
  • Added latency over the wireless NAO-camera connection (2.3s vs 85ms for webcam)

Where it fits

Governance type
university research group
Scale
lab prototype, single research team
Income level
upper-middle-income

Replication kit

Reusable artefacts from this practice — as published by their sources.

Do you run this practice? Claim it — verified implementers get a public contact pathway and can propose corrections.

Data sources

Where this practice's information was retrieved from, and when.

Attachments

Similar practices you may find useful