Alertness assistant for driver state monitoring using deep neural networks
The project proposes an alertness assist system for drivers that is capable of proficiently detecting drowsy, fatigued and sleepy drivers based on drivers? visual characteristics of eyes and cardiac activities, with the feature of generating alerts and warnings when necessary. The function of th
2025-06-28 16:30:11 - Adil Khan
Alertness assistant for driver state monitoring using deep neural networks
Project Area of Specialization Artificial IntelligenceProject SummaryThe project proposes an alertness assist system for drivers that is capable of proficiently detecting drowsy, fatigued and sleepy drivers based on drivers’ visual characteristics of eyes and cardiac activities, with the feature of generating alerts and warnings when necessary.
The function of the orchestrated system is to detect facial features i.e. eyes from the frames captured by a camera module attached to the vehicle while a subject is driving and the transfer the extracted data to the already trained model which will process the sequence of frames to detect if the driver is drowsy or alert or a warning needs to be generated or not.
The second part it plays is to keep track of driver’s cardiac activity i.e. BPM, which may get effected due to drowsiness, stress, hyper-alertness etc. the system will take mean of the initial BPM measurements by the embedded sensor and then that certain units subtracted by that mean will be considered as the threshold. Whenever the driver’s BPM measurement will be less than the thresho0ld selected, drowsiness alert would be generated.
Once the subject will show signs of sleepiness, tiredness or drowsiness or the driver will experience sessions of micro-sleep, the driver will be warned through an alert system using the speakers in the vehicle so that the person may take a break before continuing the further journey and fatal crashes could be avoided that would lead to physical and economical loss.
Our end product will be a device capable of real time surveillance of drowsy, torpid and ill-eligible driving state on the basis of visual and physiological variables The idiosyncrasy of our product is that it pledges:
Crash avoidance mitigation
By ensuring the accuracy of the proposed end system, precision in detection of torpid or ill-eligible driving state will be ensured, so that fatal car crashes could be avoided and lives would be saved.
Cost Efficacious enhance driving experience
Both Physiological measurements and computer vision techniques are least expensive in monetarily terms and most productive in performance terms. Another approach could be Vehicle-based measurements but modalities for lane detection techniques include, inertial measurement units (IMU), light detection and ranging (Lidar), geographic positioning systems (GPS) automobile dynamics, and radar, all of which require expensive sensors. So, we stick towards initial two proposed solutions, so that the product could be easily available and affordable for all types of drives such as commercial drivers who drive vehicles such as lorries, trucks, tractors, trailers, and buses and shift employees who work the night shift or long shifts and suffer from sleeping disorders.
Adaptable for all driving condition
The product will be pliable for all sorts of driving conditions and vehicle types, generic enough to be available for general public, with main goal of detecting lethargic and tired driving states.
Free advisory warning and alert system
Upon detection of driver dozy, sleepy, and heavy eyed state, a visual and acoustic warning will be provided by the system so that the driver may plunge out of sleepy zone and may take a break before continuing rest of the journey.
PROJECT METHODOLOGY
The project basically is covering two main aspects cardio logical features and visual features processing.
DROWSY DRIVER DETECTION IN VIDEO SEQUENCES USING RNN-LSTM WITH CNN FEATURES EXTRATIONS
The problematic issue in detecting driver’s drowsiness is that it is almost impossible to identify from a single frame that if the subject is confronting micro sleep periods or is blinking. In order to avoid the respective concern, our project is integrating both CNN and RNN methods which comprises of sub models the CNN model for feature extraction and LSTM for interpreting the features across consecutive frames. The technique for drowsiness detection is thus as follows:
? Significant CNN features are extracted from video frame which represent sequences of actions for certain time interval or certain consecutive frames.
? These successive frames are fed into RNN-LSTM model as an input.
? Finally, a softmax layer is used to predict drowsiness/alertness of the entire video sequence.
DROWSY DRIVER DETECTION USING PHOTOPLETHYSMOGRAPHY
In cardio logical feature, PPG sensors used utilizing motion tolerant technology, measure heart rate even during extreme physical activity. It is a simple yet momentous optical procedure which is used to identify volumetric volumetric fluctuations in the circumferential blood circulation. With this technique BPM is predicted from the surface of derma making it a least expensive and non-invasive approach. This method delivers us with crucial data about subject’s cardiac activity.
PPG BASED DETECTION
Detection techniques based upon PPG sensor have various benefits over conventional ECG based approaches. As, PPG sensor have simple hardware executions in less expense. For receiving real time readings, a single sensor need to be placed on fingertip or any other part. PPG sensor is also very cheap as compared to other sensors. The configuration and setting for other sensors such as ECG is also very complex, which make them impractical for real time operations. While PPG sensor is appropriate for handy and portable systems with less power consumption requirements.
The principle with which beats per minute are calculated is called photo-plethysmography. As the change in light intensity reflected back to the sensor from vascular region is estimated. When BPM/cardiac activity has to be monitored in real time applications, the time duration of the waves send and reflected back play an important role in estimation of the beats. Signal pulses after reflecting from the vessels and arteries are received back. Rate of pulses and rate of light absorption indicated the number of beats.
CONTRIBUTION TO THE SOCIETY
In Morocco during year 2012 a study was directed which, showed distressing numbers of fatal crashes and accidents linked to sleepiness. And these are accountable every single year for more than five thousand deaths. Not only this, but it also causes estimated substantial damage of worth 13 billion MAD.
In 36.8% of cases, lethargy behind the wheels was observed and sleepiness at the wheel was witnessed, in 31.1% of cases, comprising one forth through the proceedings of study. Moreover, it was confessed by 42.4% of the investigated drivers that they did not obey and respect the precautions of rest every 2 to 3 hours of drive.The problem which grounds social and economic mutilation in society and has become a source of apprehension for society, we as members of society have a believe that it is right to focalize on this situation and device an effective system to inhibit the commencement of lassitude and diminish the number of car crashes to lessen damages and decease rate in these type of road misfortunes.
Our project can be bifurcated into three different sections, first one including the training of a deep learning Conv-LSTM model with appropriate accuracy for identification of torpid and drowsy driving state, second one comprising of orchestrating of PPG sensor MAX30100 and defining a logic for BPM based drowsiness detection and the last one embracing the real time implementation and amalgamation of both the detection logics.
We have already covered more than 60 percent of the milestones, by covering the two major portions of formulating logic for both PPG based and Visual based recognition systems. Both of them encapsulate literature review of research papers, detailed scrutiny of prior studies regarding this domain of interest, articulating basic logic and orchestration of coding for them, reanalyzing the logics based on accuracy and precision by varying dependent parameters and in the end finalizing the best possible approach.
COMPLETED AND REMAINING TASKS
After the accomplishment of more than sixty percent of the milestones, we will be looking forward to fusion of both logics in real time, so that we would be able to device a system that would be dependent upon two features. Wrong predictions and deficiency of performance in any of the feature could get compensated with the other feature. So, our final product will be confiding on two dependencies instead of one, which will demonstrate significant improvement in the execution and results.
We have already implemented both the propositions independently in the first half course of the project timeline. And now our remaining tasks include fusing both the features and testing the model in real time.
| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Total in (Rs) | 28430 | |||
| MAX 30100 sensor | Equipment | 1 | 750 | 750 |
| arduino uno | Equipment | 1 | 680 | 680 |
| Nvidia jetson nano development board | Equipment | 1 | 22000 | 22000 |
| fitness band | Equipment | 1 | 1500 | 1500 |
| small mini wifi camera | Equipment | 1 | 3500 | 3500 |