Isharay
A mobile application which helps deaf people communicate with other people by working as a translator from sign language to text and vice versa using mobile camera and 3d modelling.
Project Title
Isharay
Project Area of Specialization
Project Summary
A mobile application which helps deaf people communicate with other people by working as a translator from sign language to text and vice versa using mobile camera and 3d modelling.
Project Objectives
- Converting sign language to text.
- Converting text to sign language.
Project Implementation Method
PHASE1:
Conversion of Sign language to text.
DATA COLLECTION:
Collection of video data of words/phrases in sign language from different websites.
DATA CLEANING:
Organizing those videos into a single usable format with unified frames per seconds (fps) for our model.
KEYPOINT DETECTION:
Detection of hand motion and collection of 21 keypoints of each hand in a single frame.
DATA TRANSLATION:
Converting the above mentioned keypoints into vectors for our machine learning model.
MODEL TRAINING:
Training of model on these vectors to classify videos against words/phrases of sign language.
PHASE2:
Conversion of text to sign language.
3D MODEL:
Making of a 3D Human Model to demonstrate sign language words/phrases.
ANIMATION:
Animating the model in sign language gestures against the words/phrases (text) from user.
PUBLIC PORTAL:
Allowing users to add new words in the vocabulory considering the regional differences in sign language.
Benefits of the Project
- Diminishing the barrier between normal and deaf people .
- Making the lives of hearing-impaired more independent.
- Creating job opportunities for deaf people.
- Making the learning of sign language easy for deaf people.
Technical Details of Final Deliverable
A mobile application that will convert text to sign language and sign language to text using mobile device.
The keypoint detection is being done by using a TensorFlow model in C++
After that these keypoints are converted to a vector which will be used to train a model in python using a neural network.
This model will be used in the mobile device or cloud to convert the sign language video to text
The 3D model will be used to convert text to sign language gestures by animation of the model.
There will also be an option to add more sign language words/phrases by user to overcome the regional differences of sign language.
Final Deliverable of the Project
Core Industry
Other Industries
Core Technology
Other Technologies
Sustainable Development Goals
Required Resources
| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| External Docker for Graphic Card EXP GDC Beast External Graphics Card Dock | Equipment | 1 | 20000 | 20000 |
| Nvidia GTX 1070 | Equipment | 1 | 50000 | 50000 |
| Total in (Rs) | 70000 |
If you need this project, please contact me on contact@adikhanofficial.com
Development of Industrial DCS for Controlling Speed of IOT based Three...
In order to control speed of three phase induction motor a Programmable Logic Control (PLC...
Water Distribution System using PLC
The UN Sustainable Development Summit 2015 proved to be a land mark by approving a plan fo...
Online election system
We are creating and developing online election system to prevent rigging and make the syst...
Development and Characterization of Novel Bioceramic based Root Canal...
Root canal therapy holds great importance in the world of dentistry and was first introduc...
Emporium Mall - Lost and Found Portal
The mobile app will be free to sign up for. This app will be available across all Smartpho...