Real Time Lane Detection and Collision Avoidance Method For Autonomous Vehicle

Project Title

Real Time Lane Detection and Collision Avoidance Method For Autonomous Vehicle

Project Area of Specialization Electrical/Electronic EngineeringProject Summary

Due to rapid advances in AI and associated technologies, cars are eventually poised to evolve into autonomous robots entrusted with human lives and bring about a diverse socio-economic impact. However, for these cars to become a functional reality, they need to be equipped with perception and cognition to tackle high-pressure real-life scenarios, arrive at suitable decisions, and always take appropriate and safest action. In the modern era, vehicles are focused on automation so that human drivers can drive at ease. As a result of how careless people drive, and the higher possibility of harmful incidents, autonomous vehicles were invented.

The project aims at ensuring the safety of autonomous vehicles by adding safety features. Drivers will be alerted if they have this feature, which can prevent them from causing collisions when they are not aware. This can make vehicle driving easier for everyone and much safer than before. The focus of this project is on two main applications of autonomous vehicles. Overview of system diagram is shown below:

Figure 1: Block Diagram

The first one is the real-time lane detection method in which the vehicle automatically follows the lane. For this, modern image processing techniques are use, Machine learning as well to replace expensive sensors like Light Radar and Radar. In lane detection Lanes and edges are detected using Open-CV Python.

Second is the real-time collision avoidance method in which vehicles will detect objects and avoid collisions. As these vehicles sense their environment and make decisions without external assistance, accuracy is vital for producing a beneficial outcome. To overcome this problem requires such an algorithm which is a module of deep learning because the system has to train to predict objects accurately. 

Project Objectives

1. To perform Preprocessing techniques on images to find road lane, Preprocessing of data is done using OpenCV algorithms.
2. To detect the lanes of the road then isolate the pixels on the left and right lanes to overwrap and display the space between the lanes.
3. To detect objects on the road in front of the vehicle for this purpose the processing is carried out on the laptop. In the vehicle's dashboard, the camera will continuously record the road ahead. The YOLOv3 development detects objects and predicts bounding boxes based on their location. Bounding boxes represent the objects of interest, including pedestrians and cars. The user or driver can also monitor algorithm performance. 
4. To train a neural network for object detection. In training a model, the goal is to improve its performance, which can be accomplished using transfer learning. An existing truck-detecting model might be useful for training another model to detect cars. Moreover, the dataset generated for this project was improved since it included images taken from various angles and bounding boxes that failed to capture the object well. Training the model was done in Google Colab.
5. To avoid collision with the objects by generating an alert or buzzer sound which depends on the specific distance from the target vehicle.

Project Implementation Method

This project involves a large amount of computational power and various techniques, like computer vision, computational neural network (CNN) and image preprocessing etc. our project can be divided into two parts, lane detection, and object avoidance. For lane detection, the image is captured from a Raspberry Pi camera and then pre-processing techniques are applied. Below shown the diagram of Lane detection Architecture:

Figure 2: Lane Detection Architecture 

Preprocessing of data for lane detection  is done using OpenCV algorithms, such as the collection of data (video), the video to frame conversion, the mapping of frames to regions of interest the classification of the data, and the frames to video conversion. The OpenCV program converts videos into frames after they have been acquired. Once the videos have been converted to frames, the data gathered in the form of images are classified for preprocessing. Preprocessing involves the following steps:

1. In the first step, converted frames are read by the program to process the new image, which is then used by the program for further pre-processing techniques to detect the lanes.
2. Next, we have to undistort the image by calibrating the distortion of the camera to remove the noise and distortion from an image.
3.  In the third step we get the perspective transform of the image. Using perspective transformation, we can see road lanes from a bird's eye view, which simplifies further processing by removing any unnecessary background information from the distorted image.
4.  In order to identify the  yellow and white pixels in the image, We have to use color mask to the perspective transform.
5. The Sobel filters are applied in addition to color masks to ensure that edges are detected. As Sobel filters are more robust to variations in color and lighting, we apply them to the Lighting and Saturation channels of images for the detection of discontinuities in the pixels along the axis of frames.
6. Finally, we combined candidate pixels from Sobel filters and color masks to obtain possible regions for lanes. The flow chart of these steps is shown below:

Figure 3: Lane Finding Operations

After these steps, we isolate the pixels of left and right lane to overlay and display the road lanes.

For object detection and collision avoidance, we use you only look once (YOLO). The YOLO algorithm is an object detection algorithm that is incredibly fast and accurate. Our input image is fed into a CNN which generates a volume of 19x19x5x85 dimensions, where each grid is 19x19 with 5 boxes. The object is detected in front of the vehicle through the camera in real-time, which then becomes the input of CNN. Therefore, CNN gives us the required output. Once we have spotted an object, we will need to determine the distance between the object and our vehicle. If the distance is too short, an alert should be generated. Block diagram of YOLO Architecture is given below:

Figure 4: YOLO Architecture

Benefits of the Project

Autonomous vehicles are expected to produce important benefits across multiple health-related domains. These include crash prevention, emissions reduction, increased mobility (and therefore quality of life) for those unable to drive, stress reduction and increased safety for cyclists. Although these are substantial health issues that contribute to the burden of disease and require enhanced prevention strategies:

Prevention of road accidents:

To make traveling more comfortable and safer, this project aims to increase road safety. The adoption of this project will save the lives of many people, as it should be cost-effective and accessible to all. Self-driving cars can reduce the driver's mistake, as high percent of crashes are caused by driver behavior or error. It’s estimated self-driving cars can reduce accidents by up to 90%.

Cost-efficient:

In this project modern image processing and machine learning techniques are used to replace the expensive sensors like Light Radar which results in a better cost-efficient method for lane detection. Compared with other sensors, the camera is now more accurate and more cost-effective at detecting objects

High Accuracy:

. Object detection algorithm based on deep learning becomes an essential method in autonomous driving because it can achieve high detection accuracy with less computing resources.

Technical Details of Final Deliverable

Our project contains the Raspberry-Pi microcontroller as a processing chip. The pi-camera module along with an ultrasonic sensor is used to provide necessary data from the real world to the raspberry-pi which contains the algorithm for object detection and also the preprocessing techniques for lane detection. Then, we deployed all the trained weights of CNN module in microcontroller which detects the objects and display results in Pi-LCD. Pi camera should be attached to the dashboard of the vehicle which captures the video of the lane that video samples can be accessed by the Raspberry-pi where all the processing happens, and the output result shown in the Pi-LCD.  In this way, our system detects the lane and objects and can reach the given destination safely and intelligently, thus avoiding the risk of human errors by responding to the real time traffic and obstacles. This would prove out to be a boon in the automobile industry as it would help in reducing the concentration required and strain put up on brain while driving also minimizing the probability of accidents due to careless or disobedient driving. The result for preprocessing of Lane is given below:

Figure 5: Input and Gray Image

Using Perspective Transform, we can see road lanes from a bird's eye view, which simplifies further processing by removing any unnecessary background information from the distorted image.

Figure 6: Perspective Transform 

the HSV space performs better since it breaks down colors into individual colors (Hue), amounts of colors, and brightness value.

Figure 7: Color Masks

We need to identify the sharpest edges in the whole frame. Several trials and errors led us to select the gradient magnitudes along the x and y directions using thresholds between 50 to 200 as the best choice for identifying edges.

Figure 8: Sobel Filter

Figure 9: Combine Sobel & Color Masks

To detect lane lines, a histogram is generated, which identifies areas that have more collections of white pixels in the image. We should aim for two peaks when drawing the histogram. The left peak will correspond to the left lane line and the right peak to the right lane line.

Figure 10: Histogram corresponding to Lanes

When no peaks are detected, a centered window is placed at the location, which is computed by shifting a previous window by a predetermined offset. We obtain the following windows and lanes.

Figure 11: Sliding Windows

To find the best quadratic, fit to the curve, we must isolate the pixels for the left and right lanes. The approximate lanes are represented by ten contiguous lines. We did so to test outlier rejection based on full lane width.

Figure 12: Isolating pixels for Lanes

As shown below, after computing the lanes, the lanes are projected onto the original undistorted image.

Figure 13:Output Lanes

Some initial video results of lane and object detection module is given below:

https://drive.google.com/drive/folders/1RVS_huwT3AewtctWLvfkxPFu5k9CFPps?usp=sharing

Final Deliverable of the Project Hardware SystemCore Industry TransportationOther Industries Others , Health Core Technology OthersOther Technologies Robotics, Big DataSustainable Development Goals Good Health and Well-Being for People, Decent Work and Economic Growth, Industry, Innovation and InfrastructureRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	53350
Raspberry pi	Equipment	1	35000	35000
Connecting wires	Equipment	5	150	750
Ultrasonic senor HC SR04	Equipment	3	200	600
Pi LCD	Equipment	1	8000	8000
Pi Camera 8MP	Equipment	1	9000	9000