Visual and Infrared Image Fusion For Security and Surveillance Applications

Project Title

Visual and Infrared Image Fusion For Security and Surveillance Applications

Project Area of Specialization Electrical/Electronic EngineeringProject Summary

The aim of infrared and visible image fusion is to obtain an integrated image that contains obvious object information and high spatial resolution background information. The integrated image is more conducive for a human or a machine to understand and mine the information contained in the image. To attain this purpose, a fusion algorithm will be proposed.

When imaging the same scene using both infrared (IR) and visible imaging systems (VIS), more information can be obtained, but more redundant information is also generated. The infrared sensor acquires the thermal radiation information of the objects in an image, therefore, the object can also be detected when the lighting conditions are not good. However, the image acquired through the IR sensor has low spectral information (Greyscale images). The images obtained through visible light sensors have more spectral information, clear texture details, and higher spatial resolution. The scene can be described more accurately by integrating the infrared and visible images into one visible-IR fused image. In the fused image, more information of the scene is available as compared to the individual original images. In addition to this, most of the unwanted/redundant information can be removed.

Infrared and visible image fusion techniques are widely applied in video surveillance, object detection, and object tracking applications. In Figure 1 the resultant fused image contains the vital details of both the Visible and IR images.

Project Objectives

1. Design an algorithm for the VIIF system.
2. Implement the working prototype which may implement the above fusion algorithm in real-time.
3. Performance improvement (previously proposed techniques lose image information which is available in the original images, fused images may contain the blocking artifacts, in brightness or lowlight (during night time) affects the performance of the fusion algorithm.)

Project Implementation Method

1. IR and Visible Image: First we obtain IR and visible images of the same scene using visible and IR cameras.
2. Image Acquisition/Preprocessing: Visual characteristics of both IR and visible images are represented in the digitally encoded representation. A preprocessing algorithm is applied for better image quality.
3. Image Registration: Due to the positional deviation of images acquired by multiple sensors (Camera), it is necessary to do image registration before image fusion.
4. Feature Extraction: Features of both the IR and visual images are extracted after image registration; these extracted features have an important role in the fused image.
5. Fusion Algorithm: We apply the fusion algorithm on both the IR and Visible Image to obtain a fused image that will contain more details of the scene.
6. Fusion Rule: The fusion rule helps the processor to decide which image features take part in the fused image.
7. Prototype Development: After applying the above steps and performing experiments, we will then design the working prototype for the security and surveillance applications. The working prototype may be a surveillance robot or a top-mounted platform.

Benefits of the Project

The project is beneficial for security and surveillance applications. It will be used by law enforcement organizations. It will be useful In bad weather and bad light conditions.

Technical Details of Final Deliverable

Explanation of Fusion Algorithm.

Block Diagram of VIIF.

Final Deliverable of the Project HW/SW integrated systemCore Industry SecurityOther Industries Medical Core Technology OthersOther Technologies RoboticsSustainable Development Goals Industry, Innovation and Infrastructure, Sustainable Cities and CommunitiesRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	37000
Cameras	Equipment	2	5000	10000
Arduino	Equipment	1	1500	1500
Raspberry PI	Equipment	1	9000	9000
Motors	Equipment	5	500	2500
Supply	Equipment	1	2000	2000
Mechanical Structure	Equipment	1	5000	5000
Misc	Miscellaneous	1	5000	5000
PCB Printing	Miscellaneous	1	2000	2000