Toll Tax collection using OpenALPR

Project Title

Toll Tax collection using OpenALPR

Project Area of Specialization Artificial IntelligenceProject Summary

OpenALPR is an image processing library built in C++ using OpenCV functions. OpenALPR is used for number plate detection and character extraction. The objective of this project is to design a mockup toll tax collection system by using OpenALPR and pre trained models for vehicle body type classification. The system is implemented on toll booth where the toll booth operator have to select vehicle image for further analysis. OpenALPR performs basic steps on the source image and gives us segmented characters. Later on these segmented characters are compared with character tiles using structural similarity index. This gives us the probability of each character and lastly characters with greater probability are stored in a string which is the actual number plate of vehicle. Likewise imageAI, retina net model and pre-trained Yolo darknet weight file is used to classify the vehicle in three categories i.e. car, bus and truck. The resulting data is then used to determine vehicle toll tax based on vehicle body type. The system is implemented and simulated in python and tested on a real image. It is observed from testing that our mockup system is successful.

Project Objectives

This is a research based project and following are mine current objective of the project:

• To avail vehicle body classification details using image processing.
• To fetch vehicle number plate data including single line and multi line numberplate using OpenALPR.
• determine vehicle toll tax based on vehicle number plate and vehicle body type classification.

Project Implementation Method

Doing this type of project all alone and with no background experience of machine learning and computer vision was a big ask for me, but fortunately with proper guidance of mine teachers and supervisor i did'nt  backed out. i faced a lot of difficulties while building this project.

mine first and formost task was to run openalpr, unfortunately the C# binding available for opelalpr were of .exe format i.e. they cant be used for development purposes. i tried alot of other bindings as well like c++ and java, but lastly i got succeeded in runing its python binding. the next step was to train the algorthm for Pakistani number plates. Our number plates are based on provinces and each province has thier own standards and style. for example federal number plates are with white backgound, Sindh number plates are with yellow background, Punjab number plates has two portions i.e. green and white. than there are single line and multiline numberplates. so in short they require a lot of dataset for training purposes. so mine second step was to train the openALPR detector. this requires 2000+ images. so i contacted NHA and requested them to allow me to make dataset, which they approved. so i adjusted the camera at certain angle at chenab toll plaza and lastly collected images out of the video. than i annotated these images  and trained the software through haar cascade classifier as recommended by openalpr. unfortunately the dataset was not of that great quality, so i was facing lot of problem in optical character recognition. at last after trying several methadolgies we decided to make character tile similar to that segmented character which we get form openalpr and compare these two together using structural similarity index. this gives us the probabilty of each comapred charcter and characters with high probability are stored in a string which gives the number plate of the vehicle. for vehicle body type classification i am using imageAI, retinanet model and yolo trained weight file. 

Benefits of the Project

Every project has its own pro's and con's. This project of mine is based on current industrial problem, so i think it can help a lot in that regard. following are the benefits of mine project.

•  This project of mine can used as mockup system for toll tax collection and with further research and analysis it can be used as proper alternative system.
• a very big benefit of our project is that it can be used for security purposes with slight modification. let's suppose a vehicle is stolen, if we install this system than we can easily trace the vehicle with the help of its number plate. we have to just search the respective number plate in the database and the system will tell us about when that particular vehicle was at which location.
• A very big advantage of the project  is that it can be used easily on the current system. there is no need for buying new infrastructure e.g. cameras, computers server etc.
• If we compare our project as a system with current system implemented at toll plazas, this project has much lower maintainability cost.
• this project can be used for toll tax collection and also numerous other applications like security surveillance, parking systems, secured locations like housing societies where they have to maintain the in and out details of vehicle.

Technical Details of Final Deliverable

there are number of project online and people are working on this type of project but i didn't find any study or project which can detect Pakistan's single line and multi line number plates. 

Our project can classify vehicle into three main categories i.e. car,bus and truck using image. it has the ability to detect and read Pakistan's provinces number plate which include both single line and multi line number plates. there still is room for plenty of research and development.

Final Deliverable of the Project Software SystemCore Industry TransportationOther Industries Telecommunication Core Technology Artificial Intelligence(AI)Other Technologies OthersSustainable Development Goals Industry, Innovation and InfrastructureRequired Resources