Deep Learning & Computer Vision● ACTIVE RESEARCHIHub-Data IIIT Hyderabad DST India
Design and development of communication system for the speech and hearing disorder persons
Project Summary
“Developing a model to do real time conversion of Indian Sign Language(ISL) to sentence.”
RESEARCH GRANTRs. 24,59,4242025 – 2027
RESEARCH OVERVIEW & DETAILS
Project Scope & Details
Developing a model to do real time conversion of Indian Sign Language(ISL) to sentence.
ARCHITECTURAL & SCIENTIFIC HIGHLIGHTS
01.
Multi-Modal Hand & Upper-Body Pose Estimation
Simultaneously tracks 21 hand joints, facial expression blendshapes, and shoulder posture at 60 FPS.
02.
Continuous Sign Language Transformer
Attention-based sequence-to-sequence model translating continuous visual sign sequences into syntactically correct text.
03.
Spatial Grammar & Non-Manual Marker Modeling
Decodes head tilts, eyebrow furrowing, and spatial directional verbs unique to Indian Sign Language (ISL) grammar.
04.
Real-Time Bi-Directional Speech Synthesis
Converts recognized signs into synthesized spoken audio while providing live captions for inclusive communication.
PROJECT GALLERY [1 PHOTOS]
Click to expandFIG. 01
Gesture Recording
RESEARCH TEAM
- Dr. Himangshu Sarma
- Dhanvanth Singari
- Aditya Pal
- Anudeep Tippabathuni
- Dwara Gowri Sri Prabha
- Abhilash Surapuchetty
- Susrith Dasari
- Sambi Reddy Kalumuri
- Mamatha Jonnalagadda
EXTERNAL COLLABORATORS
- ▸IHub-Data IIIT Hyderabad DST India
- ▸IDEAS, ISI Kolkata
FUNDING & GRANTS
SANCTIONED BUDGETRs. 24,59,424
SPONSORING AGENCYIHub-Data IIIT Hyderabad DST India, DST, Govt of India
PROJECT TIMELINE2025-02-01 – 2027-03-01
TECHNOLOGIES
Deep LearningComputer VisionTransfer Learning on Vision ModelsHolistic Pose DetectionTransformersGenerative Models