Pavement Health Monitoring using VTOL System

This is a project to accommodate the inspection process of roads in terms of time and efficiency. It is related to detecting cracks in the streets while producing a summarized report of that road via a drone and a cloud server on which a machine learning model will be deployed. The basic idea is to

Project Title

Pavement Health Monitoring using VTOL System

Project Area of Specialization

Artificial Intelligence

Project Summary

This is a project to accommodate the inspection process of roads in terms of time and efficiency. It is related to detecting cracks in the streets while producing a summarized report of that road via a drone and a cloud server on which a machine learning model will be deployed. The basic idea is to speed up the process of road inspection, which is currently being done manually where a surveyor goes to the site location and painstakingly draws the road along with all of its cracks and other defects, which consumes a lot of time, human effort and money.

Fortunately, this project aims to provide a digital solution to this problem. A drone will be equipped with components such as a high-resolution camera and a Single Board Computer like a Raspberry Pi. Additionally, the drone can be piloted over the road at a certain height. These components will be responsible for imaging the road and transferring them to a cloud platform, where those images will be processed via machine learning models and computer vision techniques to generate data regarding cracks locations on the roads. This data will then be processed and sent to a cloud database to display the data on a particular platform, such as a mobile app. In addition to this, the collected data will be available for report generation as long as the user has access to the application and has a stable internet connection.

Project Objectives

Problem Statement:

Poorly maintained roads cause accidents in a variety of ways, mostly due to the fact that they create an enormous hazard to drivers. In many instances, a driver may attempt to avoid a certain situation, like a pothole or pooling water, which could cause a serious accident. In the United Kingdom, The government spends handsomely on the road networks and the roads individually take a healthy amount of time to be made or repaired manually by utilizing the labor.

Aim of the project:

Considering the time taken to inspect manually is quite high which becomes an inconvenience to the localites, modern technology shall be utilized to optimize the process, decrease the meantime to repair and be less of a disruption to the citizens. This project provides the solution to the aforementioned problem. The aim of the project is to build such a drone that can:

1. Inspect roads and detect road crack

2. Process and analyze the images of road cracks

3. Display and report the data extracted from the analysis

This can be achieved by utilizing an online backend server which acts like a bridge between the database and the frontend. For frontend, a mobile application or a website is to be availed by users where the data will be displayed. Hence, this project meets the requirements of  National/Local Road Maintenance Authorities or Highway Maintenance Authorities who are responsible for maintenance and repair of road networks of large areas. In this way, the drone will replace the labor involved in manual surveying of the roads, crack detection and transferring data to concerned authorities for further analysis.

Project Implementation Method

This process involves five main components:

1. Drones
   Parts of Drone
   The drone selected for our project is a quadcopter of F450 frame. The drone has been broken down to each component and listed in the list below:

   No.	Material	Quantity
   1	F450 Frame	1
   2	ESC-30A	4
   3	BLDC motor	4
   4	10 inch propeller	4
   5	XT60 connector	4
   6	Naze32 Flight Controller	1
   7	FS-iA6 Receiver	1
   8	FS-i6 Transmitter	1
   9	Raspberry Pi	1
   10	Gimbal & Camera	1
   11	USB Port	1
   12	WiFi Dongle	1
   13	SD Port	1
   14	Two port push button	1

2. Cloud Server

   For the backend, we will be using a dual backend system where one will be dependent on NodeJS and the other one will be Python based. NodeJS based backend will be the one we deploy on Firebase itself for the socket functionalities provided by the firebase with its Firestore service and Realtime Database service where we can trigger a method when data is added, deleted or updated in the database. It is quite handy in automating the whole process since addition of images can indicate the project being started and it can point towards the cloud to run machine learning algorithms on the images to get the results as soon as possible. On the other hand, Python’s backend will be created using the Flask library which will have the whole machine learning algorithm programmed in it and will be deployed on a virtual machine since no service has python backend support in Firebase.

3. Machine Learning
   Dataset:
   Our dataset consists of four types of road damages/cracks:
   - Longitudinal (parallel) cracks annotated as D00
   - Transverse (Perpendicular) cracks annotated as D10
   - Alligator (Complex) cracks annotated as D20
   - Potholes annotated as D40

         
   Figure: A longitudinal crack                                                         Figure: A transverse crack                                Figure: An alligator crack                                                                      Figure: A pothole
   
   Object Detection:
   Object detection is a Deep learning method that gives us the ability to locate and classify objects in a digital image. With this type of segmentation, object detection algorithms can be used to calculate the number of detected objects in a frame or video and determine their accurate locations in an image, all while accurately labeling them.
    

4. Database:

   We will be using NoSQL over SQL as NoSQL provides various benefits over SQL, including the scalability and flexibility. NoSQL is basically a collection and documents based database in which collections of similar documents are formed and each document consists of related information.
   

No.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Benefits of the Project

Cracks and potholes have been a huge structural problem on the roads. The common causes of pavement deterioration and degradation are overloading, seepage, improper or poor road surface drainage, lack of proper road maintenance, lack of proper design, adverse climatic conditions and some other factors. Road distresses disturb and adversely affect the traffic flow and traffic safety leading to poor performance of the road. They also cause an increase in fuel costs, result in time delay and prove troublesome for every road user.

This is a step towards Industrial Automation and provides benefits in four major domains:

1. Greater Efficiency - Road cracks can be detected more efficiently and accurately. Furthermore, the location and the image of the cracks will be available on the application as well providing a backup document for the cracks.

2. Increased Productivity - It increases the overall productivity as our proposed system not only automatically detects the cracks but also provides accurate location and image of the cracks which are saved on the application in the form of reports.

3. Human Safety - This provides safety to the human labour as they do not need to walk through the roads to track the cracks, rather it will be done automatically by the drones while humans only control the drone from one point.

4. Time Saving - It saves the time which is occupied by attempting this process through human labour. Also, it prevents the citizens from the hustle when roads are blocked for longer time to detect cracks manually.

Technical Details of Final Deliverable

1. Drone Assembly
   The table below shows the components, which are so far, integrated in the drone assembly phase:

   No.	Material	Quantity
   1	F450 Frame	1
   2	ESC-30A	4
   3	1000KVA BLDC motor	4
   4	10 inch propeller	4
   5	XT60 connector	4
   6	Naze32 Flight Controller	1
   7	FS-iA6 Receiver	1
   8	FS-i6 Transmitter	1

2. Data Collection

The dataset used to train the road damage detection model consists of 26,620 annotated road cracks and damage images belonging to four classes taken with a mobile phone camera from multiple countries. In the training data, the total number of instances for different types of damages are: 

 

Figure: Different types of damages

3. ML Model Results

The figures below show cracks detected in Pakistani roads:

The figures below show cracks detected in Japanese roads:

The figures below show cracks detected in Indian roads:

The figure below shows cracks detected in Czech roads:

4 Database:

We selected Firebase as our main database and storage point. Firebase is a product of Google which provides various features including Firestore and Realtime Database that are for textual data storage while cloud storage is also provided which can store any non-textual data, which includes: images, videos and any other format of data

Our structure will be such that we will have 2 collections in total; one will be of users that are registered and other one will be of projects. When a project is created, the project document will be added to this and this document may have a collection for defects found on the road in this project. This nested collection will show the relation of these defects to this project so it can be easily managed and displayed.

5 Mobile App

The app will have a basic flow where at first when the app is launched, a person is asked for credentials if he is already a user or not, then he can register on the application and login after that. Once a person is logged in a screen with all of his recent projects will appear where he will be able to visit through his recent project or create a new project if he wants to. Project creation button will lead to a form which includes all the necessary detail required in order to create a project including its title and further basic information. On clicking a project, ‘Project’ dashboard will appear where you can see all the necessary features and other stuff that are needed to compose a report for a project and to display all the data gathered from images by machine learning model.

No.

1

2

3

4

5

6

7

8

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Manufacturing

Other Industries

Core Technology

Artificial Intelligence(AI)

Other Technologies

Sustainable Development Goals

Decent Work and Economic Growth, Industry, Innovation and Infrastructure, Sustainable Cities and Communities

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com