Gaze Controlled Desktop
This project aims at proposing such a system that can replace Keyboard and Mouse and control desktop using only eye movement. This system will be useful for both physically disabled and able persons as it is focused on enhancing the human computer interaction which it will, in both the cases. Existi
2025-06-28 16:27:27 - Adil Khan
Gaze Controlled Desktop
Project Area of Specialization Artificial IntelligenceProject SummaryThis project aims at proposing such a system that can replace Keyboard and Mouse and control desktop using only eye movement. This system will be useful for both physically disabled and able persons as it is focused on enhancing the human computer interaction which it will, in both the cases. Existing systems that use eye gaze gestures for controlling desktop involves the use of hardware device (sensors and special cameras combined). We here propose a system that can control the desktop using eye gaze gestures just as the existing systems but by using only the webcam, thus making it more feasible and affordable.
Gaze Controlled Desktop is mostly focused on people who have a physical disability and cannot make use of their hands. They’ll be able to perform basic desktop control operations (like keyboard and mouse control) using only their eye movement.
Project Objectives- Replace the traditional keyboard with an on-screen keyboard from which the user can type by fixating gaze on keys.
- Replace the traditional mouse by using gaze to hover the mouse cursor and performing mouse click events by eye blinking.
- Adjust brightness levels and sound volume from the onscreen menu. Navigate using the arrow keys from the menu. The menu will also contain icons for selecting keyboard and mouse.
- Turn screen on/off automatically by detecting the presence of the user.
- Perform additional functions like drag and drop, scrolling, zoom in, zoom out.
The system will be developed using Incremental model. Every module will be designed, developed and tested hand in hand so that we achieve maximum efficiency and make the system possibly error free. Moreover, a backend module will be developed using OpenCv and CNN Face detection algorithms from Dlib combined with the Eye Gaze Datasets that will be trained using deep learning on Google Colab altogether implemented on Visual Studio. A webcam will be used for the input using Image processing. A desktop application will be developed as a user interface.
Python: For developing different modules
OpenCV: Human face libraries and algorithms used for face detection and then eye movement detection.
Deep Learning (Google Colab): Using python for training data sets.
Eye Gaze data sets: To develop a trained module that can detect gaze gestures.
Visual Studio: Used for developing the front end user interface and the back end combined with OpenCv and Dlib.
Dlib: CNN algorithm for Face detection and eye movement tracking.
Web Cam: Used as an input device using image processing.
Benefits of the ProjectWe ought to design a system that allows a person to use the desktop with the help of eye movement and move the cursor as per the direction that is given by the person. Aim of this application is to introduce a low cost based system for the people who have the physical disability and connect them with the real world.
In order to provide more convenient and natural communication mechanism for the user to interact with the computer, we propose a system that uses eye gaze gestures to control desktop and replaces the traditional keyboard and mouse thus providing a better Human Computer Interaction for both the disabled and the able people without using any external hardware component. All that our system needs is a webcam which is already present in most modern laptops and monitors.
Technical Details of Final DeliverableThe following modules will be the main final deliverables of our project:
Keyboard Control: Replaces the traditional keyboard with an on-screen keyboard from which the user can type by fixating gaze on keys.
Mouse Control: Replaces the traditional mouse by using gaze to hover the mouse cursor and performing mouse click events by eye blinking.
On-screen Menu: Adjust brightness levels and sound volume from the on-screen menu. Navigate using the arrow keys from the menu. The menu will also contain icons for selecting keyboard and mouse.
Presence Detection: Turn screen on/off automatically by detecting the presence of the user.
Additional Functions: Perform additional functions like drag and drop, scrolling, zoom in, zoom out.
To summarize, the basic deliverables of the project would be:
- A desktop application for the interaction of the user.
- An on-screen menu displayed after the application has started.
- A backend module that uses trained datasets of eye gaze in combination with OpenCv, Human face libraries and certain algorithms for the detection of face and eye gaze
| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Total in (Rs) | 52840 | |||
| Logitech C310 webcam | Equipment | 1 | 7840 | 7840 |
| SP Silicon Power Ace A55 SSD 256GB | Equipment | 1 | 5500 | 5500 |
| 8GB DDR3 Laptop RAM | Equipment | 1 | 4500 | 4500 |
| ASUS GEFORCE GT 710 4GB GRAPHICS CARD (GT710-SL-2GD5-BRK) | Equipment | 1 | 25000 | 25000 |
| gaze pointer research subscription | Miscellaneous | 1 | 10000 | 10000 |