Social Glasses for Deaf using Augmented Reality

The proposed system has two ends, i.e., a mobile application end and a hardware end. The hardware consists of augmented glasses headset which provide captions on the screen of the glasses. Augmented text will be shown as the subjects speak, on the Glasses Display. This sense of emersion will help th

Project Title

Social Glasses for Deaf using Augmented Reality

Project Area of Specialization

Augmented and Virtual Reality

Project Summary

The proposed system has two ends, i.e., a mobile application end and a hardware end. The hardware consists of augmented glasses headset which provide captions on the screen of the glasses. Augmented text will be shown as the subjects speak, on the Glasses Display. This sense of emersion will help the deaf and mute be able to be a part of society with real-time translation through portable AR glasses headset.

The system will display text accurately by first converting the audio into speech and then the text. Once the text is found, the system will start detecting the speaker by checking the video for any person. Afterward, the system will associate the person with the text on a successful match between them both.

The system is also having emotions as emoji icons according to the different voice tones of the speaker’s speech. Once detected the expressions of the speaker will be augmented along with the speech text.

Similarly, the app can detect a user's friends which are saved by them with their pictures and other details. If the system successfully detects the speaker as a friend by comparing their facial features, then it will augment their details with the associated person.

Project Objectives

• To build a system for deaf and mute people to help them with communication.
• Digitally capturing the sound and displaying the user's needed information in the augmented text is the main motive of this project.
• Removing the barrier between deaf and normal person so that be a part of society.
• Using state of the art technology because it is demanding in near future.
• Providing free app for benefiting society.

Project Implementation Method

Agile Development Methodology, Scrum in Specific.

Software is based on iterative and incremental processes due to which Scrum works best. Scrum is an adaptable, fast, flexible, and effective agile framework. It supports teamwork and maintains project history(backlog) and is ideal for our group of 3 members. We are using Trello for maintaining our project. Daily meetings are conducted, and tasks are evaluated based on the sprint backlog. After each module, a new sprint is made.

Benefits of the Project

The system can be used by the public as a means to socialize in the future. Our system has the capability to be used by the wide public by introducing features of social media into their daily lives. Once successful in its mission to help socialize the deaf, features such as chat, meetings, navigation, reminders, games, advertisements, etc. can all be merged into this augmented system.

So far, there is no reliable solution for the deaf to get sound information in real-time so that they can easily interact with daily life. Such a system will create the initial funding for its growth. The go-to-market strategy can be achieved as this system will deal with all the communication gaps that arise with the deaf by encouraging other developers to further add new features.

It is our responsibility to give back to society, and this is how society lives. The Social Glasses are an initiative to help the deaf community around the globe, be able to use the prospects of technology for the betterment of their lost abilities and eventually help society as a whole.

Technical Details of Final Deliverable

AR Glasses Headset

AR Glasses Headset use the phone’s hardware to process the image that is shown on the glasses screen.

Application

The Application is multiplatform and works on the AR glasses as well as mobile applications for Android & IOS platforms. It consists of 4 core functionalities:

Speech to Text Augmentation

Voice Emotion Detection

Alerts System

Friend/Identity Detection

There are three modes of Social Glasses:

• Mobile
• VR
• AR

Document

Complete FYP Documentation, inclusive of all that is mentioned above.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Others

Other Industries

IT

Core Technology

Augmented & Virtual Reality

Other Technologies

Artificial Intelligence(AI), 3D/4D Printing

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com