Real Time Three Dimensional Reconstruction using Kinect Fusion

Real-time 3D Reconstruction using Kinect Fusion uses a Kinect sensor and a gyroscope to scan real-world objects and make their interactive 3D models in real-time. With the growing real-time applications of robotics in agriculture, industries, and homes with respect to mapping the environ

Project Title

Real Time Three Dimensional Reconstruction using Kinect Fusion

Project Area of Specialization

Augmented and Virtual Reality

Project Summary

Real-time 3D Reconstruction using Kinect Fusion uses a Kinect sensor and a gyroscope to scan real-world objects and make their interactive 3D models in real-time.

With the growing real-time applications of robotics in agriculture, industries, and homes with respect to mapping the environment and interacting with it, traditional 2D images provided insufficient data to meet the requirements which is why we need 3D reconstruction. 3D reconstruction gives 3D information of the scanned environment where the focus of our project will be on scanning objects. These scanned objects can then be 3D printed, manipulated in design software such as AutoCAD and SOLIDWORKS, or used as data set for machine learning algorithms. 

Previous techniques (Multiview Stereo, Structure from Motion, and Monocular RGB) used to perform 3D Reconstruction were either inefficient or insufficient as

• They were not in real-time.
• The reconstructions were rudimentary.
• They were light-dependent.
• They could not scan smooth surfaces.

Our project aims to perform 3D reconstruction in real-time using easily available, cost-effective equipment to make it more accessible in the future. 

Project Objectives

The objective of our project is straightforward. It is to attempt to resolve the downsides of previous techniques and perform 3D Reconstruction which is

• Dense and detailed
• In real-time
• Able to detect smooth surfaces
• Able to scan object in variable lighting conditions 

Project Implementation Method

Our project will be implemented in the following steps

1. Interfacing the Kinect with software development kit (SDK)
2. Selecting a suitable programming environment and language
3. Displaying Kinect input data streams to ensure proper hardware functioning
4. Reading RGB and Depth image from the Kinect 
5. Converting RGB and Depth image to Point Clouds
6. Camera tracking using camera calibration, iterative closest point algorithm, and gyro-sensor data to fuse point clouds from all angles together into a 3D point cloud
7. Perform volumetric integration which is to join points in the point cloud into triangles
8. Perform raycasting to apply texture and make a 3D model of the scanned object

Benefits of the Project

3D reconstruction is one of the main areas of research for Computer vision. Computer Vision is a field of artificial intelligence that trains computers to interpret and understand the visual or the natural world as we humans understand and perceive it. If we can teach our robots to identify as accurately as possible the environment around them, then we can teach them how to interact with it.

Achieving the level of interaction with the environment which is similar to that of humans is a key step for the advancement of self-driving cars, to use robots as nurses in the hospitals, to compensate for the declining workforce by using robots for harvesting, to avoid working in dangerous conditions by using robots to mine minerals and to save human lives by performing accurate and precise surgeries.

Technical Details of Final Deliverable

A running environment that will reconstruct the object scanned through the Kinect in real-time. The environment will enable users to 

• Adjust depth of the scanned area
• Display RGB and Depth Image of the scene
• Display live reconstruction of the scanned object
• Create a mesh (.stl file) of the scanned object in real-time

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Others

Other Industries

Core Technology

Augmented & Virtual Reality

Other Technologies

Sustainable Development Goals

Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com