Gestures Talk

The framework is made to support the hard of hearing and quiet network to effortlessly convey and communicate with their close by encompassing.System includes two modules. First module is a hand glove with flex sensors, gyro/accelerometer sensor and ARDUINO to convert hand gestures to digital data.

Project Title

Gestures Talk

Project Area of Specialization

Robotics

Project Summary

The framework is made to support the hard of hearing and quiet network to effortlessly convey and communicate with their close by encompassing.System includes two modules. First module is a hand glove with flex sensors, gyro/accelerometer sensor and ARDUINO to convert hand gestures to digital data. This digital data is sent to android APP via Bluetooth module. Second module is an Android App which have eight modes.

Mode 1: Convert Gesture digital data into ASL (Alphabetically).

Mode 2: Convert Gesture digital data into English Phrases.

Mode 3: Convert English Phrases into ASL.

Mode 4: Learning exercises in real-time by using glove.

Mode 5: Number system compatibility.

Mode 6: Addition of local languages (Urdu, Punjabi etc.)

Mode 7: Smart view (Screen-Mirroring) in application.

Mode 8: Enhanced application with appealing interface.

Project Objectives

The project came into existence for one sole purpose, to help the deaf community to easily communicate and interact with their nearby surroundings. The aim is to convert basic symbols that represent the 26 English alphabets as mentioned under ASL (American Sign Language) script and display them on a smartphone screen and convert gesture into basic phrases.

Project Implementation Method

The system is proposed with the use of flex sensors and android technology with the help of unity framework.The smart glove containing flex sensor and tilt sensor was previously used as a controller in video games. The glove sends the raw data to the processor, according to the raw data some action was taken by the processor.  Smart glove was also used to capture the gesture and translate the English alphabet according to ASL but as only the translation of alphabets not help to reduce the communication barrier between mute deaf community we introduce a complete embedded system which contain a smart glove to translate ASL into audio phrases and text phrase to ASL.

Benefits of the Project

There are 2.78%[1] of the total population of Pakistan who can’t speak. Sign language is a nonverbal form of communication method which is found among all deaf and dumb communities in world. Normal people do not learn the sign language. It causes barrier in communication between deaf dumb and normal people. Hence Deaf-Mute communication required interpreter who will convert hand gestures into auditory speech. The past implementation of this project involved using image processing concept and accelerometer. But the drawback of these implementations are projects were non portable and too expensive.Our Project will act like a bridge.

Technical Details of Final Deliverable

There are 2 components in this project. First component base on glove and second component is mobile application base. First component is based on Raspberry Pi for connectivity requirements. 10 flex sensors and accelerometer are connected Raspberry for calculating the sensors values. Second component based on mobile application. It shows gesture message according to the value sent by glove.And Second it Shows Gestures according to the text in respect of ASL.

Final Deliverable of the Project

Hardware System

Core Industry

IT

Other Industries

Education , Telecommunication

Core Technology

Robotics

Other Technologies

Sustainable Development Goals

Reduced Inequality

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com