Design and Implementation of Gesture Control Algorithm for Quadcopter using Vision Sensor with Communication Link

Over the past few years the Unmanned Aerial Vehicles (UAVs) have rapidly grown in popularity. Offering an increased work efficiency and productivity, the drones have become an important focus in various applications including agricultural monitoring, disaster management, surveillance, remote sensing

Project Title

Design and Implementation of Gesture Control Algorithm for Quadcopter using Vision Sensor with Communication Link

Project Area of Specialization

Robotics

Project Summary

Over the past few years the Unmanned Aerial Vehicles (UAVs) have rapidly grown in popularity. Offering an increased work efficiency and productivity, the drones have become an important focus in various applications including agricultural monitoring, disaster management, surveillance, remote sensing, target acquisition, border patrol, infrastructure monitoring, photography and videography. Moreover, UAVs are a useful tool for researchers to test and evaluate new ideas in a number of different fields, including flight controls theory, navigation, real time systems and robotics. In the past decade, comprehensive efforts were made to make the UAVs fully autonomous; however, flying an UAV is still a quite challenging task. Gestures are the non-verbal means of communication, in the modern era gesture recognition is the main interest in the field of computer vision and image processing. Motion of hand can also perform some gestures such as move left or right etc. With this the need of hand detection come into play. To date, hand detection in the complex or cluttered background is considered a challenging task.

The idea of the proposed project is to design a fully autonomous quadcopter that can be controlled using the hand motion and gestures. Our proposed algorithm will have the basic competence and intelligence to detect and track humans’ hand even in the cluttered background using modern computer vision technology. The detection will be on the ground-based camera i.e. laptop camera and the corresponding action will be done on the flight controller which is mounted over the quadcopter with a capability to reach near to real time implementation. The downlink for the live video streaming will also be implemented using a communication protocol.

For autonomous hand detection a data set of 4000 images are collected and trained by using Viola-Jones object detection technique [1]. After this, the trained model is capable of detecting palm with the accuracy of 85%. From the detected area the minimum eigen features are extracted and tracked using Kanade-Lucas-Tomasi (KLT) tracker [2]. For a robust tracking, minimum of 100 feature points are required otherwise, the Viola-Jones object detector will again detect the palm. To make the proposed algorithem more robust, we have employed Kalman Filter to predict the hands trajectory [3] in future frames.

[1]          P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” Proceedings of the 2001 IEEE Computer Society Conference on ComputerzVision and Pattern Recognition. CVPR 2001.

[2]           J. Shi and Tomasi, “Good features to track,” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94, 1994.

[3]          Weng, Shiuh-Ku, et al. “Video Object Tracking Using Adaptive Kalman Filter.” Journal of Visual Communication and Image Representation, vol. 17, no. 6, 2006, pp. 1190–1208., doi:10.1016/j.jvcir.2006.03.004.

Project Objectives

• Design and implementation of a robust and effective hand detection algorithm based on OpenCV.
• Design and implementation of real time hand tracking algorithm based on Kanade-Lucas-Tomasi tracker and Kalman filter.
• Design and Assembly of the quadcopter, including the integration of the flight controller, microcontroller, motors and camera.
• Design and  implementation of the communication protocol between on-ground station (laptop) and on-board flight controller using MAVLink.
• Design and development of the control algorithm to effectively track the target with high accuracy.
• Optimization of the hand tracking algorithm to achieve real-time implementation.
• Complete integration of the hardware and software portions for development of the complete prototype.

Project Implementation Method

• Hardware
  • Quadcopter design calculation for thrust and weight
  • Selection of hardware components based on design calculations
  • Assembly of quadcopter
  • Autonomous take-off and landing
  • Integration of camera for live video streaming
  • Transmission of video stream to display via video transceiver.
• Software
  • Feature based tracking using KLT tracker
  • Machine Learning based technique for object detection using cascade classifier
  • Collection of Hand Dataset to train cascade classifier
  • Implementation of Kalman filter to achieve robust tracking
  • Implementation of MAVLink protocol to communicate between ground base station and quadcopter
• Hardware and software integration
  • Generation of commands for quadcopter motion
  • Transmission of command for left, right motion etc. via Telemetry,

Benefits of the Project

The main benefits of the project are listed below

• Fully autonomous, machine learning based remote less quadcopter which can be operated by untrained users with higher flying accuracy. This can be employed for various applications including photography and videography.
• It will increase the safety of flying manifold.
• It will reduce workload of the users by allowing UAV movement using hand gestures only. 
• Can be used in strategic areas where user’s movement is restricted.

Technical Details of Final Deliverable

• A fully functional autonomous quadcopter.
• Robust and effective hand detection and tracking algorithm.
• A fully functional communication protocol based on MAVLink to send flight commands from ground station to the flight controller.
• A fully functional downlink for the real time video streaming.
• A journal or a conference paper. Along with the BS thesis report.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Education

Other Industries

Core Technology

Robotics

Other Technologies

Sustainable Development Goals

Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com