Communication System for People with Locked In Syndrome

Project Title

Project Area of Specialization

Artificial Intelligence

Project Summary

Locked-in syndrome is a rare neurological disorder in which there is complete loss of all voluntary movements of muscles except for the eye. Similar to Paralysis, it is the loss of voluntary muscle movement. Our project will enable the patients of this disease to commuincate with those around them. We will be developing a communication system for people suffering from locked-in syndrome which will allow these patients to be able to express themselves. Our project will allow the patients to control this software with their eyes. They will be able to control the software and communicate with the people around them and express themselves simply by blinking their eyes.

Project Objectives

Eyes will be detected
Eye mapping will be detected
Eye blinks will be converted into text
User will be able to communicate with those around them

Project Implementation Method

The laptop will be placed at a safe distance from the patient and the software will be started. Desktop will be placed at such a position where the patient’s eyes will be visible. Patient will be able to see rows of alphabets. One row will be highlighted at a time. After some time the next row will be highlighted. This will be repeated until the patient blinks intentionally. When the patient blinks intentionally, that row will be selected. Then in that row, each of the characters will be highlighted one at a time until the patient intentionally blinks. And then that letter will appear on screen. A beep sound will be made to tell the patient that their blink has been detected. It will also predict what the patient is trying to write based on the software’s history. Then the patient will have the option to recite what he’s typed on the desktop screen.

We will differentiate between intentional and unintentional blinks by determining how long does a blink last. If a blink lasts more than a determined time, say 1.5 seconds, then it will be intentional otherwise it will be discarded as unintentional.

Eye blinks can be detected in three ways. One of them being detecting the Eye Blinks using facial landmarks. We can use Dlib to detect facial landmarks in Python. Then we can use the Eye Aspect Ratio Equation to determine whether the eyes are open or closed. It will give a constant value when the eyes are open. And 0 when the eyes will be closed.

Some gestures related to the eye will be mapped in the software which will control the software. The gestures will be taught to the software to be recognized using machine learning.

Benefits of the Project

People with Locked-in Syndrome will be able to communicate with everyone around them
People with other similar forms of paralysis will be able to communicate witht those around them
These people who were not able to do much will be able to do simple things like ordering food from websites and order other miscellaneous items according to their needs