Sign Language Translation Platform: Automated Speech-to-Sign Production and Sign-to-Speech Recognition Using Media Pipe Technology

DOI

https://www.doi.org/10.59256/ijire.20260702051

Published In

Volume 7, Issue 2 (2026)

Published Online

March-April 2026

Pages

437-443

Article Type

Original Article

Check for updates

Sign language plays an important role in bridging the gap between hearing-impaired and hearing people. However, the lack of knowledge of sign language creates a communication gap between the deaf community and the general population. Our project aims to reduce this communication barrier and strengthen the connection between them. The proposed system is a two-way communication system that converts sign language into text or speech and vice versa. The system uses a camera to capture hand and body movements and converts them into text or speech. It also uses machine learning to convert text into sign language animations. The system is user-friendly and privacy-friendly, as all processing happens on the user's device. The project works by capturing live or recorded video through a camera and detecting hand, body, and face landmarks using Media Pipe. These landmarks are then processed and classified using TensorFlow.js, which recognizes sign gestures and converts them into text or speech. The reverse process, text-to-sign conversion, is performed using CDL3, which detects the language and translates text into sign language. A 3D avatar is used to display the results as sign gestures. This project uses machine learning to improve communication accessibility and provides fast and accurate translations. In the future, the system can be improved by increasing accuracy and supporting more sign languages