Image-based Real-time Passenger Overloaded Vehicles Detection using Deep Learning

Project Title

Image-based Real-time Passenger Overloaded Vehicles Detection using Deep Learning

Project Area of Specialization Artificial IntelligenceProject Summary

We propose real-time image-based overloaded passenger vehicles detection using deep learning. The motivation behind this project is to reduce the risk of accidents that are mostly caused by overloaded vehicles. In addition, overloaded vehicles cause damages to the roads. The detection of overloaded vehicles is difficult due to the variations caused by several factors. Especially in an environment like Pakistan, detecting overloaded vehicles is still a challenging task due to visual variations, cluttered background, camera orientation, obstacles, and illumination problems. Similarly, variations in vehicles shapes and heights also make it difficult to recognize overloaded vehicles. Consequently, we propose a deep learning-based approach for overloaded vehicles detection. An image dataset of overloaded vehicles such as overloaded buses, vans and other vehicle types is collected and pre-processed. This dataset is then manually annotated to generate ground truth vehicles that are overloaded with passengers. This dataset is then split into training and test sets in order to train and test the deep learning algorithms. The training set along with the annotations are used to train state-of-the-art object detection algorithm YOLO (You Only Look Once) more particularly YOLOv3 and YOLOv4. The test set is then used to evaluate the overloaded passenger vehicles detection accuracies of both YOLOv3 and YOLOv4 which are 90% and 93% respectively. For real-time implementation, the trained model is then deployed on a Raspberry Pi that captures the road video with its Pi-camera and performs real-time detection of overloaded vehicles.

Project Objectives

The following are the objectives of our proposed system: 

• Dataset Collection of Passenger Overloaded Vehicles.
•  Dataset Annotation.
•  Performance evaluation of deep learning-based object detection algorithm namely YOLO in OpenCV and Python.
• Real-time deployment of the framework using Raspberry Pi

Project Implementation Method

The following are the steps for implementation:

1. Passenger overloaded and non-overloaded vehicles image data collection: The images of overloaded and non-overloaded vehicles are collected from roads as well as from the internet. The dataset consists of Passenger overloaded and non-overloaded vehicles images. The total number of images in the dataset is 4200 out of which 2100 are of overloaded passenger vehicles whereas 2100 are those of non-overloaded passenger vehicles.
2.  Images pre-processing: The collected image dataset is pre-processed such as cropping, resizing and aspect ratio correction. The unnecessary image parts such as flat roads and sidewalks are cropped out. The cropped images are then corrected for aspect ratio in order to bring them to a standard aspect ratio such as 4:5. These images are then resized to 614x614 pixels before input to the next step which is annotation.
3.  Image annotation for ground truth generation: In order to train and then test the deep learning model, a ground truth of these images needs to be generated. This is done via the manual annotation of images in “LabelImg” by drawing bounding boxes around the overloaded and non overloaded passenger vehicles. Our dataset consists of two classes, labelled as overloaded vehicles and non- overloaded vehicles. These annotation are saved as separate files for each of the images.
4. Performance evaluation of deep learning models: The pre-processed image dataset is then used to train and test the object detection framework. We evaluate two variants of YOLO which are YOLOv3 and YOLOv4. The dataset is randomly split into training and test sets where the training set is 85% of the whole dataset whereas the rest of 15% constitutes the test set. The training set along with the annotations are used to train the deep learning model and the test set is used to evaluate its performance for overloaded vehicles detection.
5. Raspberry Pi Implementation: The trained model is then deployed on a Raspberry Pi for real-time implementation. The Raspberry Pi captures the video in real-time with it Pi Camera which is then fed to the framework that performs real-time detection of overloaded vehicles.

Block Diagram:

Benefits of the Project

The project scope targets the support for law enforcement authorities detect the overloaded vehicles. Consequently, following are the benefits to the current scenario:

• Automatic detection of overloaded vehicle on roads thereby reducing the labor and costs involved in road patrolling
• Reduce the risk of road accidents due to vehicles overloading thus saving precious lives
• Making the roads safe to travel as overloaded vehicles can also damage the roads
• Raspberry Pi device to perform detection of passenger overloaded vehicles and non-overloaded vehicles in one place.

Technical Details of Final Deliverable

The final deliverable will include the following:

• A processed and annotated dataset of 4200 images of overloaded and non-overloaded passenger vehicles.
• Trained deep learning models of YOLOv3 and YOLOv4 that can perform overloaded vehicle detection
• A Raspberry Pi where these trained models are deployed with an interfaced camera and Python code running to perform the
• whole operation of the framework

Final Deliverable of the Project HW/SW integrated systemCore Industry TransportationOther Industries Education , IT Core Technology Artificial Intelligence(AI)Other TechnologiesSustainable Development Goals Decent Work and Economic Growth, Industry, Innovation and Infrastructure, Responsible Consumption and ProductionRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	56000
Raspberry Pi 4 - 4GB	Equipment	1	35000	35000
Camera Module for Pi	Equipment	1	9000	9000
Card Reader	Equipment	1	500	500
SD Card 32 GB	Equipment	1	1500	1500
Other Expenses(Documentation, Printing, Posters)	Miscellaneous	1	10000	10000