SINTHEA

People who are visually challenged or partly blind might benefit from smart assistive eyewear. A camera, a number of sensors attached to it, and a portable computing device for processing the data would make up these smart glasses. Furthermore, Deep Learning and Artificial Intelligence will be used

Project Title

SINTHEA

Project Area of Specialization

Artificial Intelligence

Project Summary

People who are visually challenged or partly blind might benefit from smart assistive eyewear. A camera, a number of sensors attached to it, and a portable computing device for processing the data would make up these smart glasses. Furthermore, Deep Learning and Artificial Intelligence will be used to identify things with high accuracy and low latency, allowing the user to have a real-time experience of their surroundings while walking. In Pakistan, around 2.5 to 3 million individuals are blind, and they rely on traditional sticks for aid, which leads to numerous issues and mishaps due to a lack of intuition about their surroundings. To solve this issue, we devised a plan to commercialise the production of glasses for this purpose. To solve this issue, we devised a method for commercially producing spectacles for the visually handicapped or partially blind. This concept and solution will undoubtedly attract a large number of consumers from all over Pakistan. We will not only be carrying out a business effort, but also contributing to society and assisting those with vision impairment through our enterprise. Transferlearning models are considered efficient alternatives to the laborious process of training a deep learning model from scratch in order to handle this challenge.These smart glasses will be designed for all kinds of applications and contribute in the areas of information technology, business, logistics, medical and healthcare, security, and government, etc., the fundamental objective of the proposed smart glasses is to blend human vision with the virtual world, producing a kind of ubiquitous computing. Using this design, the camera placed on the device obtains the video equivalent to the point of view of the person which will then be further processed for assisting the visually impaired user. Alongwith that, there wont be any need for the user to hold the device or move it in a certain position in order to make it properly, providing ease of-use and comfort to the user. Furthermore, we propose to design and develop an extension to this prototype by making in work in daytime with much more efficiency. Lastly, we also propose to develop a system, allowing the smart glasses to localize itself, guide the user and navigate them towards their destination through path planning and voice assistance 

Project Objectives

Our goal is to help blind people and bring them to the normal folds of life by enabling them to work in industries, academia and other parts of the societies. The product aims to make them a useful part of the society and also help preserve their mental, psychological and even physical health.

• Detect a certain objects in the live video stream coming from the mounted camera.
•  Estimate the distance between the blind person and each object.
• Take the audio input from the user through a microphone.
• Notify the user through audio output about that specific object.
• Recognize the person standing in front of the user.
• Tell the user through audio output the name of the person.
• Help navigate the blind through in door obstacles safely and successfully using audio commands.
• Operate in  daytime  .

Project Implementation Method

This project will use cutting-edge deep learning methods to create trained models that can recognise items. YOLO, a Convolutional Neural Network (CNN) for performing object localization gradually, is one of the suggested algorithms. CNNs are classifier-based frameworks that may interact with input images as ordered types of data and discern between designs. YOLO has the advantage of being much faster than other organisations while yet maintaining accuracy. It enables the model to look at the complete picture at test time, allowing its forecasts to be influenced by the global context in the image. YOLO and other convolutional neural organisation computations assign a "score" to regions based on their resemblance to predetermined categories. 

High-scoring locations are identified as the specific location of the class to which they are most closely related. For example, in a live feed of article location, YOLO may be used to identify various sorts of goods based on which areas of the video score are most connected with predetermined item classifications.

 For the purpose of determining the distance from known object(s) to camera, we will use triangle inequality. First, we check the resolution of our webcam and then find its fixed focal length. Here a Logitech C270 webcam is used as a sample.

 ???? = 1430         (1) 

(1) is a fixed focal length of the webcam and we assign this value to focal length in our implementation script

. ???? = (???? ???? ????)/????     (2)

In above equation (2), D is distance, W is known width, and B is box-width. We apply this equation in our project and the result was quite promising.

In addition, we'll use APIs to integrate voice assistance into the smart glasses. This is especially significant because the blind and visually handicapped rely on the rest of their senses for the most part, and hearing is by far the most crucial of them. The glasses will be able to recognise and detect a variety of things, including humans, but not exclusively. The voice API will get the needed information from the computer vision/image processing section of the glasses, which will then disseminate it to the user. Similarly, the speech API will process the user's commands, and the glasses will be prompted to carry out the instructions, such as finding a certain location around the user.

Benefits of the Project

Around 2.5 to 3 million people are blind in Pakistan whereas, 2.7% are visually impaired and this ratio is increasing day by day. This results in the increasing demand for such smart glasses, that allow the visually impaired to avail this accessibility device at a low cost.

The current crop of smart glasses that are designed for blind and visually-impaired people is very extravagant and not affordable for an average person in our society. They do not provide the functionalities that should be in them, because those functionalities are very indispensable to facilitate the life of a blind person or impaired person. By seeing all those shortcomings in existing products, our product will provide them at an affordable rate.

A study aimed to offer estimates, trends, and forecasts of Pakistan's visual loss burden from 1990 to 2025. As of 2017, around 1.12 million (95 percent) of Pakistan's 207.7 million people were blind (Visual Acuity [VA] 3/60), 1.09 million [0.93–1.24] people had severe vision loss (3/60VA6/60), and 6.79 million [6.00–7.74] people had moderate vision loss (6/60VA6/18). With 552.98 YLDs [392.98–752.95] per 100,000, Pakistan is placed fourth among other South Asian countries and twenty-first among the other 42 low-middle-income countries (as defined by the World Bank). Blindness and visual impairment among all-age YLDs increased by 55 percent in 2017 compared to 1990, the tenth largest rise among major health loss causes (such as nutrition), iron deficiency, headache problems, and low back discomfort are common. These figures reveal that by 2025, the burden of vision loss would have increased, indicating that Pakistan will need to enhance its efforts to combat the rising incidence of eye illnesses. All of this translates to a largely unexplored market in Pakistan with few rivals.

When we decide to put our product on the market (go for commercialization) after conducting all of the necessary research and surveys. We are convinced that our product will be a boon to job creation, and that we will be able to capture the whole market in our demographics, giving an inexpensive and effective solution to every blind or visually impaired individual.

Technical Details of Final Deliverable

This project is based on state-of-the-art deep learning algorithms, comprising of trained models which will detect objects in real time. For this project we will be using one of the Convolutional Neural Network (CNN) algorithm,CNNs are classifier-based frameworks that can interact with images as ordered collections of data and discriminate between them. YOLO, which is a CNN for performing object detection and localization, For this project we will be using Yolo version 5 (v5), which is one of the latest algorithm in the YOLO family and has a very good accuracy rate and fast object detection speed. YOLO detects object in an Image frame and localize the object in an Image frame by drawing a bounding box over it. And Identifying the class and telling as well as its confidence level.

We'll utilise triangle inequality to figure out how far away known object(s) are from the camera. We first evaluate our webcam's resolution before determining its fixed focal length.

Furthermore, we will be incorporating voice-assistance through APIs into the smart glasses. This is especially important because of the fact that the blind and visually-impaired depend for the most part on the rest of their senses and the ability to hear is the far and foremost among them.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Medical

Other Industries

Manufacturing , Health , Security

Core Technology

Artificial Intelligence(AI)

Other Technologies

Augmented & Virtual Reality, Wearables and Implantables

Sustainable Development Goals

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com