An Intellgient Life-Logging System for Alzheimer patients using BCI based Smart Glasses

Project Title

An Intellgient Life-Logging System for Alzheimer patients using BCI based Smart Glasses

Project Area of Specialization Computer ScienceProject Summary

Today, doctor’s availability for each patient turns to be a non-viable, where doctors are outnumbered to number of patients. People urge towards new technology, using smart methods which gives accurate results similar to the doctors. But still there are many neural disorders which requires a lot of time and doctor’s attention for treatment. Patients of dementia or Alzheimer requires regular medical assistance, such patients are usually cared by family members or relatives. The demanding attention for most of the task can limit a caregiver’s ability to help themselves, as it's very exhaustive and tiresome job and increase anxiety, depression in those people. Some families hire someone as a governess for those patients but it also increase pressure on their budget. To overcome this problem, technical experts took initiatives in smart health system and invented smart wearable gadgets which are intelligent enough to treat those patients without external help. Smart gadgets like talking alarm clocks, remote reminder clock, different GPS location and tracking devices and home care robots which helps for Alzheimer’s patient at some extent. Our proposed intelligent system helps Dementia patients in troublesome situations and help them connecting back to their lives. We provide an advance wearable system based on wireless technology which is more portable, ease the patient to use it. This project is to monitor the patient’s emotional behavior and by their happy, sad, and confused feelings it respond appropriately to make patients feel better in real-time. Our project is using a BCI headset to detect user brain signal and feeling predictions and MADGAZE smart glasses to capture real time photos as well as showing them on its see-through display. In the same way, it will apply deep learning algorithms and image processing on captured photos for analyzing and predicting person/s in those picture and assist patient to remember those person. Brain emotional activities of the patients and the captured images will be stored in a real-time database secured for unauthorized access, like doctor can only read emotional status of the patient to check depression level.

Project Objectives

To develop a intelligent, efficient, and user-friendly system which will assist dementia patients, our main objectives are:

• To build BCI based smart glasses where life logging is implemented on the basis on patient’s emotional state.
• An effective system that virtually display captured scene images to the users on see-through display of glasses when required.
• A system will assists the patients in daily life tasks and make them feel independant of other's help.
• A system which will use deep learning models to help patients in recalling process when they are trying to identify someone.
• To develop an intelligent device which will capture real time video or images as memories of patient.
• To smartly record only joyful memories of the patient instead of recording every movement.
• Developing a cloud-based system for backup of storage and ensure security of patient's data and also to provide analysis of data in real time.
• Making a wireless system to relax the patient and let them roam anywhere while wearing them.
• Developing intelligent system with user friendly interface.
• Purposed system will also reduce the cost of hiring nurses for taking care.
• Deploy different learning algorithms for the testing of unseen data and train the model to predict desired results.

Project Implementation Method

System Architecture:

This architecture for our model is shown in figure 1. The basic components of architecture are discussed as follows:

1      MADGAZE Ares:

Madgaze ares will be used to communicate the resultant images to the patient on see-through display. This will also capture videos whenever instructed to do so.

2      Emotive EPOC+:

EMOTIV EPOC+ is the BCI Interface that detects brain signals and extract information from these signals after processing which will be further sent to the cloud. These device will instruct the MAD GAZE smart glasses for further execution after acquirin required signals from brain.

3      Cloud Storage

After performing basic operations on acquired images, Long-time data processing and storage is done with the help of cloud services. The processed images received from the server is saved on the cloud separately for future use. The cloud data is secured so that only authorized users could access patients.

Benefits of the Project

Following are the benefits provided by our proposed system to patients and the whole domain.

• In this project life logging is happening intelligentaly based on patient's mental state.
• It captures real time images or videos intelligently, which can be replayed for reviving patient's memory.  
• This project is beneficial in such a way where dementia patient can do there daily life tasks independently.
• It is cost effective, where we dont need to pay for governess's services.
• This project will also boost pateint's moral and connect them back to the life.
• It will also helpful for dementia patient in recalling process,where the detail description of a specific person will be displyed on the smart glass's screen.
• This system is wirelessly connected, so patient can keep it anytime at anywhere with ease.
• Only authorized users can access pateint's data as it is stored securely on cloud.
• This project will also reduce the stress of caring on pateint's family and relatives.
• This project allowing hands-free control of the system, so that the operator can simultaneously carry out other tasks.
• It will allow user a see through display where user can see what is shown on the glass screen while still being able to see through it.
• It will occupied very less storage where life logging is done smartly where unnecessary information will not be capture.

Technical Details of Final Deliverable

1      BCI based emotional state identification

In this system, we are using BCI headset emotiv epoc to detect EEG signals of brain and predict patient`s emotions. BCI is a device, collects brainwave frequencies such as delta, theta, alpha, beta, and gamma, which also indicates the brain signals can be processed in a deeper manner for emotion classification, specifically on the basis of frontal, temporal and other regions that deals with emotional experiences.  Emotiv Epoc electrode positions will be placed at those parts of brain to read brain signals and classify them after extracting features to compare them with required trained data. It sends instructions to external device when signals frequency is matched with required dataset.

2      Intelligently Capturing Images and Videos

We will be using MAD GAZE ARES smart glasses to record real time experience of the patient based on their emotional status. It gets instructions from BCI headset to perform specific tasks like start recoding or to display patients’ old happy experiences from cloud storage. On happy emotions BCI headset command smart glasses to start recording, whatever user is watching and enjoying, by using its front camera and after that it will apply image processing algorithms on it. On sad emotions, it will retrieve those images and videos from the database and show them to the see-through display of the glasses to entertain the user. On confusing state, our system will identify the persons (if any) in the image and give description of that person to patient on see-through display.

3      Image Processing on Captured Images

After capturing images and videos from smart glasses, we are applying image processing algorithms on them to reducing noise, image segmenting, feature extracting etc. and applying machine learning model on it for identifying persons in the images. Our trained model compares the extracted features of captured image with trained features and predict the person in the image as an output. The description of the person will be projected on smart glasses display further. These algorithms will be stored in the external memory of the smart glasses which will boost the whole performance of the system.

Final Deliverable of the Project HW/SW integrated systemCore Industry HealthOther Industries IT , Telecommunication Core Technology Wearables and ImplantablesOther Technologies Artificial Intelligence(AI), Internet of Things (IoT), NeuroTechSustainable Development Goals Good Health and Well-Being for People, Life on LandRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	80000
MADGAZE ARES	Equipment	1	70000	70000
Miscellaneous	Miscellaneous	1	10000	10000