millimeter Wave Radar based pedestrian and vehicle detection for Autonomous cars.

Project Title

millimeter Wave Radar based pedestrian and vehicle detection for Autonomous cars.

Project Area of Specialization Artificial IntelligenceProject Summary

The project is based on the functioning of mmWave radar AWR 1642 operating as a short range radar (SRR) for pedestrian and vehicle detection. The project includes designing of an autonomous car using two core components, AI camera and mmWave radar for driving assistance. The projectseeksto eliminate the weakness ofself driving carsto avoid accidents in bad weather, lightning and fog using a radar to detect objects: pedestrian and vehicles in its path.

Project Objectives

In order to achieve the final aim, the main objective of the project is to create a system with a combination of mmWave Radar and AI based smart camera for detection of pedestrian and cars. The data from radar will be used for identifying the type of objects it detects. Furthermore, the combination of AI camera and radar will result in a self driving car functional in every kind of environment. Our main goal with this project will be to be able to detect objects with mmWave radar and classify them using AI techniques. The AWR1642 has many applications in the automotive industry but our focus will be to assist an autonomous car in object detection and avoiding accidents.

1. Study about mmWave Radar and Al based smart camera. 2. Study about the implementation of self driving car decision making.

3. To integrate both systems to create a functional prototype of an autonomous car.

Project Implementation Method

The project is designed to complete in a phases, so that at each phase the project’s error can be eliminated and the accuracy/ efficiency of the system can be maintained for sustainable output.

1 In our first phase that is the implementation phase, the mmWave Radar and AI camera will be implemented in parallel. The Radar and Camera in this step will be tested separately for the achievement of the output desired for our system.

2 For the testing of the functionality, in the first phase the radar will be tested by observing traffic from stationary position and matching the outputs of point clouds to the output of camera. For example, point clouds will be used to predict if an object in sight is a pedestrian or a vehicle. AI camera and radar will be tested on a prototype self driving track containing roads, obstacles, road signs, traffic lights and vehicles. Blurring the camera so that objects become hard to detect with just camera will turn on radar assistance and the car will be tested under low light, fog conditions. The conditions for successful functioning would be if the car completes the drive course without hitting, crashing or getting out of track boundaries.

3 After the complete and functional implementation of AI camera and point segmentation radar, they will be integrated into a car prototype using a Raspberry pi module. The Raspberry pi will send data to a work station for 3D plots and the camera image. Other autonomous car parameters such as lane assist, object detection and steering information will also be sent to the workstation for further processing. This is how we will successfully carry out the project for our satisfaction.

Benefits of the Project

There are several advantages of using mmWave Radar in an Autonomous Car. The mmWave Radar is not only just reliabile while driving autonomous car but also extract some useful data from the detected object. The following are the advantages of using mmWave Radar.

(i). Robust to bad weather and Darkness: Unlike AI camera, the mmWave Radar is immune to the unfriendly weather like massive rain, fog and darkness that usually present at the countryside and rural areas.

(ii). Speed detection and Motion sensing: The mmWave Radar has the capability to measure the speed of detected object such as the vehicle and pedestrians. It also has the ability to estimate the traffic volume of different transportation modes, such as pedestrian, motorcycle and car, and estimate their average speeds. Moreover, the mmWave Radar also performs the motion sensing. Whereas, the AI camera does not have the capability to measure the speed of the detected object such as the vehicle and pedestrians and also does not has the ability to perform motion sensing.

(iii). Distance Measurement: The mmWave Radar also has the ability to measure the distance of the detected object, this feature enable the mmWave Radar to replace the Ultra-sonic sensors and because the Radar itself has the ability to measure the distance. Whereas, the AI camera does not have the ability to measure the distance the detected object such as the vehicle and pedestrians.

Technical Details of Final Deliverable

The main goal is to create a self driving car prototype which functions using the AI camera and mmWave radar AWR1642 using Short Range Radar (SRR) configuration. The main component of this project is mmWave radar which helps us to detect the objects in front of it which will assist the object collision and accidents aspect of the car. The points recorded by the radar will be further processed to identify the type of object and classify it. If we talk on software basis the programming is required on both ends for example on mmWave Radar’s firmware as well as on the AI based smart camera used for the image processing. This can be implemented using programming languages like: Python Programming. On the other end the hardware requirements are mmWave radar, camera for AI programming furthermore, if it is applied in any kind of system we will use for then, its respective hardware parts will be arranged accordingly.

1. AWR 1642 mmWave radar: Millimeter wave (mmWave) is a special class of radar technology that uses short wavelength electromagnetic waves. Radar systems transmit electromagnetic wave signals that objects in their path then reflect. By capturing the reflected signal, a radar system can determine the range, velocity and angle of the objects. MmWave radars transmit signals with a wavelength that is in the millimeter range. This is considered a short wavelength in the electromagnetic spectrum and is one of the advantages of this technology. Indeed, the size of system components such as the antennas required to process mmWave signals is small. Another advantage of short wavelengths is the high accuracy. An mmWave system operating at 76–81 GHz (with a corresponding wavelength of about 4 mm), will have the ability to detect movements that are as small as a fraction of a millimeter. TI devices implement a special class of mmWave technology called frequency modulated continuous wave (FMCW). As the name implies, FMCW radars transmit a frequency-modulated signal continuously in order to measure range as well as angle and velocity. This differs from traditional pulsed-radar systems, which transmit short pulses periodically.

2 AI Camera: The camera will be powered by a Self driving AI able to make decisions such as steering, maneuvering, brakes, reading street signs, finding alternate paths, following lanes and all the functions that a human does while driving a car. This will be the backbone of the autonomous car. Hardware will be decided as per the software needs when it is developed.

3 Raspberry Pi: This will be the core compute unit of the car. All the processing and decision making will be done on the raspberry pi. The radar and AI camera will communicate all the information to the raspberry pi and it will be responsible to deliver this information to the driving mechanism of the car. Raspberry pi will also be responsible for sending monitoring data to the cloud for further processing and optimization.

Final Deliverable of the Project HW/SW integrated systemCore Industry TelecommunicationOther Industries Transportation Core Technology Artificial Intelligence(AI)Other Technologies Internet of Things (IoT)Sustainable Development Goals Industry, Innovation and InfrastructureRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	79560
Raspberry pi module 4 (4 GB)	Equipment	1	12000	12000
AWR1642	Equipment	1	40000	40000
Arduino Uno	Equipment	1	760	760
Nvidia Nano 2 GB	Equipment	1	12000	12000
USB camera	Equipment	1	4800	4800
Car kit	Miscellaneous	1	8000	8000
Accessories and jumper wires	Miscellaneous	1	2000	2000