Real time Manipulation of Multiple Robotic Arms with hand Gestures using Kinect Sensor

Project Title

Real time Manipulation of Multiple Robotic Arms with hand Gestures using Kinect Sensor

Project Area of Specialization RoboticsProject Summary

Main Reasons:

The main reason of this project is to make a gesture controlled multiple robotic arms using kinect sensor, because human hands are not physically suitable for some harsh environments like in military, industrial environment or in radio active regions, so by our proposed system one can use the human intelligence and a robot’s physical strength at the same time to perform difficult tasks. 

Theme of the Project:

The theme of the project is that to make a human-robot interaction system using kinect sensor. The kinect will receive the image data of moving hand the person should be within the 3 meter range of the kinect sensor, then the kinect will send the data to our PC by kinect Adapter, the image data will be processed by a software called kinect SDK which will calculate the coordinate location of the human skeleton, and then we will use these coordinate locations to derive or control the positions of motors present in the robotic arms, so the robotic arm will follow the human arms..   

Need of Design:

Need of this design is two robotic arms with controllers, then a kinect sensor will be used to take video information of human gestures.

Ideas to be achieved:

The idea is to achieve a successful human robot interaction system using kinect sensor and two robotic arms

Project Objectives

The objectives of the project is as follows.

• The aim is to make the robotic arm to understand the human body language thereby building a bridge between human and the robot.
• To reduce man effords.
• To make an efficient human-robot interaction system using kinect
• To make a robot smoothly mimic the human.

Project Implementation Method

The system monitors the motion of the user's arm using a Kinect. The skeletal image of the arm obtained using the “Kinect Skeletal Image” project of Kinect SDK, consists of 3 joints and links connecting them. Coordinate Geometry is used to calculate the angles between the links connecting the joints. This gives us the angles for a 3D representation of the human arm. The angles thus obtained are sent using a serial communication port to the Arduino microcontroller, which in turn generates signals which are sent to the servo motors. The servo motors rotate according the angles given as input. The combined motion of the servos results in a complete Robotic arm movement which is a mimic of the human arm movement.

Benefits of the Project

 Robotic arm is used in various areas such as military, defense, medical surgeries, pick and place function in industrial automation applications. Based on the gesture of human hands the robotic arm moves and performs the task and this system replicates the actions of human hands. The arm is very flexible and can be made suitable in places where the environment is not safe for humans like firework manufacturing industry, bomb diffusing etc. 

Technical Details of Final Deliverable

The deliverables of the project are as follows.

• Two Robotic arms with controllers
• Kinect sensor
• Kinect adaptor for windows

We will give two fully automated robotic arms that will be framed that they will look like human arms, and we will give a kinect sensor and its adaptor to connect it to a PC.

Final Deliverable of the Project Hardware SystemCore Industry ManufacturingOther IndustriesCore Technology RoboticsOther TechnologiesSustainable Development Goals Industry, Innovation and InfrastructureRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	75000
2 Robotic Arms with Controllers	Equipment	2	25000	50000
Kinect Sensor	Equipment	1	10000	10000
Kinect Adaptor	Equipment	1	8000	8000
Project Reports & Stationary	Miscellaneous	1	7000	7000