FPGA Based Real Time Object Tracking
Visual object tracking and detection is a sub-field of computer vision and image processing. Object detection algorithms typically use extracted features and learning algorithms to recognize instances of an object category. So, first of all we detect the target object then we track them by using dif
2025-06-28 16:32:40 - Adil Khan
FPGA Based Real Time Object Tracking
Project Area of Specialization Electrical/Electronic EngineeringProject SummaryVisual object tracking and detection is a sub-field of computer vision and image processing. Object detection algorithms typically use extracted features and learning algorithms to recognize instances of an object category. So, first of all we detect the target object then we track them by using different techniques which is normally difficult to human brain to process these pictures and videos. So, its main objective is to find the locus of points that the target of interest follows in image coordinates by assigning labels in temporarily spaced video frames. This information may be of pertinent importance for further analysis such as to calculate the center of mass, area, perimeter and motion vector of the target. So, usability spectrum of visual object detection and tracking is very wide and it play a vital role in high level image analysis tasks, such as object recognition (face, car etc.), activity analysis and intelligent scene understanding.
The aims and objectives of the project are listed below:
- Video Capture/Display through camera link interface
- Algorithm optimization on MATLAB
- Implemented algorithm should work in real time, and accurately track the object
- Hardware implementation of the considered algorithm on ZYBO Z7-20 board
- Performance evaluation of the selected algorithm
Filter based trackers represent the appearance of objects to be tracked using filters trained on example images. Firstly, the target or object is initially locked or selected using the tracking window which is placed on the object in the first frame. From this point on, filtering and tracking work together. After this correlation is performed on the next frames to track the targeted object and this is selected on the value of correlation value, where the value of the correlation is high this is actually the new position of the object to be tracked. So, this update is continuously performed to track the object.
Methodology is a series set of steps to achieve a specific goal. The methodology of our project is shown in the following figure:
Video input using Arm Processor:
We use HDMI-IN port of the zybo board to take the input of the video frames. We make an IP-Design of this to take input, from which HDL wrapper is created. This HDL wrapper is actually Verilog top level file of the IP-Design. This is linked with PS (dual core ARM processor) and then ARM processor is connected with PL (Processing Logic) which is FPGA actually. This interconnection uses the AXI Bus for communication of PL and PS. To take input which is designed in IP-Design is configured in PS actually using SDK. This SDK tells PS to take input from the HDMI-IN port and send it to the PL using AXI Bus. The basic steps in this part are shown below:
Processing on FPGA for object tracking:
Tracking is performed on FPGA because hardware implementation has high speed in any perspective. So, we perform the task of tracking on FPGA. Overall, this approach is actually called hardware software co-processing. However, barematel applications have more high speed then this type but it is difficult to implement these types of projects. Because, everything is performed on hardware is tough job due its complexity and we also have to write protocols of everything we want to perform. So, we choose this method of hardware software co-processing technique to implement the correlation tracker.
HLS is known as High level synthesis and this tool is used to create the IP-Core using High level languages like C and C++. This is best tool which give us ease to create the Verilog or HDL file of our tracker. So, we implement the tracker in HLS using high level language and create the Verilog actually the IP-Core which is used in IP-Design to perform the task of tracking on FPGA. So, we can easily perform the task of tracking using HLS tool and put this IP-Core in IP-Design which give us hardware specification for the task of tracking and high efficiency. This IP-Core is also synthesized in HLS.
Displaying output using ARM Processor:
We use HDMI-OUT port of the zybo board by Xilinx to display the output frames on the monitor. This is also defined in IP-Design and interface is from PL to PS and then processor put this data on the HDMI-OUT port using the same AXI Bus.
Advantages of the project are listed below:
- Fast and efficient tracking system: As the project is based on FPGA , it will have much better FPS due to the parallel processing property of FPGA.
- Security organizations: For tracking down theSuspicious activities, security organizations can use this project.
- Useful for National Highway Athourity: NHA can also use this project for speed measurement of vehicles after some modification.
- Traffic flow analysis: To analyze the congestion on different routes at different times, such as peak hour calculations or behavior of the specific type of vehicle.
The deliverables of the project contain a complete system in which video is coming as input to the zybo7-20 board, the tracking algorithm is applied to that video for the given object and the final output of the system will be displayed on the monitor.Object tracking system require high performance computer, for real time tracking in video processing system exploiting the parallelism of reconfigurable hardware. Image and video processing-based algorithms, which implemented on FPGAs has several constraints like resource utilization, timing closures, power and speed. All the necessary hardware modules for tracking-by-detection algorithm are effectively implemented and verified. Tracking algorithm has many applications in embedded system, for rapid prototyping of video processing algorithm using FPGA.
| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Total in (Rs) | 70875 | |||
| Zybo z7-20 | Equipment | 1 | 50701 | 50701 |
| Camera | Equipment | 1 | 6474 | 6474 |
| Hdmi Cables | Equipment | 2 | 600 | 1200 |
| Prototype Case | Equipment | 1 | 4500 | 4500 |
| Miscellaneous | Miscellaneous | 1 | 8000 | 8000 |