Deaf and Dumb Communication

We are introducing a two way deaf and dumb communication, the project is building a system that assist deaf and dumb people to convey their messages to normal people and a normal person can also respond them easily without learning sign language by using our android application. We have a two wa

Project Title

Deaf and Dumb Communication

Project Area of Specialization

Internet of Things

Project Summary

We are introducing a two way deaf and dumb communication, the project is building a system that assist deaf and dumb people to convey their messages to normal people and a normal person can also respond them easily without learning sign language by using our android application.
We have a two way communication system, in this system one way of communication is for deaf and dumb person talk to a normal person, we are using Leap Motion device for hand movement recognition will be done and then by using motion track data of leap motion device, system will be trained according to Sign Language. The communication module of proposed system will acquire data through leap motion, that data will be used to recognize the sign and then convert into an audio form in the speaker.
Also, another way of communication is for a normal person to talk to a deaf and dumb person without knowing and learning sign language, we build Android Application in which the communication module of proposed system will acquire data (voice) from a normal person through microphone, data store/retrieve in clouds and then convert into text and send through application to screen in the form of text, additional features like emergency will be available in the Android Application.

Project Objectives

Deaf and Dumb conversation is hinged on sign language which renders words through hand and finger symbols. Main objective of the project is building a system that assist deaf and dumb people to convey their messages to normal people. The system consists of two main parts. The frst part is hand sign recognition into audio form and the second one is speech recognition into text form. This ensures a deaf and dumb Communication. Our project will mainly focus on the following objectives:

• The aim of this project is to bridge a communication barrier.

• A system which helps normal and deaf and dumb people to effectively communicate with each other.

• In emergency or other times when a deaf and dumb person traveling or among new people communication with nearby people or conveying a message becomes very difficult.

• Hand gestures recognition system is widely used technology for helping the deaf and dumb people.

• The aim of the research is bridge the communication barrier, we propose an system which helps normal and deaf and dumb people to e?ectively communicate with each other

Project Implementation Method

• Develop Inter-conversion technique

The project is mainly hardware based, hand gestures using PSL (Pakistani Sign Language) are recognized by the Leap Motion Device and features are extracted from Raspberry pi and converted into audio thus sound is produced from a speaker and the other normal person can respond by recording their message in android application and sending it to the screen installed on Raspberry pi. All of the hardware gadgets will be installed on Raspberry pi.

• Implement Basic 100 words

We will implement basic 100 words of Pakistani Sign Language....

• Build the Android Application

We will use Android studio to develop the Android Application for speech-to-text conversion. With the mobile application or android application we can record a message and transmit to the screen of the deaf and dumb communication hardware.

• Emergency Options and Location Detection:

We will design Android App in which emergency calls and location tracking is also available. The person in case gets lost; location can be traced by the android app user using the GPS module.

Benefits of the Project

• The caretaker/relatives/loved ones can track the person through the android app to make sure that he is safe and sound.
• The project “Deaf and Dumb Communication” is the service to the society. The main scope is to help and remove hindrance to the hearing impaired people in our society, to help them communicate.
• We will develop an Android Application for a normal person to talk to a Deaf and Dumb person, in which we will use the Google API for the sound to text conversion which will guide the Deaf and Dumb person, what the normal person is trying to say.

Technical Details of Final Deliverable

• Hardware Interface: Raspberry Pi is used which will act as a brain of the system or minicomputer on which all hardware will be installed. Raspberry Pi screen will be installed on the system for the receiving of text from a normal person.
– Sensor : Leap Motion Device Model 2

– Raspberry Pi : Model 3b

– Screen : Android Smartphone, TFT screen

– Microphone : Android Smartphone

– Android Mobile : Any

– Speaker

• Software Interface: An android mobile application is a part of the project to make two way communication possible and detect the location of the Deaf and Dumb person through the GPS module installed on Raspberry pi. Google API is used for voice to text conversion which will be accessed through the internet.

– Programming Languages : Python

– Integrated development environment : Raspbian v4.14 , Android Studio v1.0

– Cloud : ThingSpeak

– Google API : Speech to Text.

Final Deliverable of the Project

HW/SW integrated system

Type of Industry

Education , Medical

Technologies

Internet of Things (IoT)

Sustainable Development Goals

Reduced Inequality

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com