IMU based gesture controlled mobile robot

Services robots directly interact with the people, so finding a more natural and easy user interface is of fundamental importance. While previous works have been focused primarily on issues being manipulation and navigation in the environment, some robotics are used with user-friendly interfaces

Project Title

IMU based gesture controlled mobile robot

Project Area of Specialization

Robotics

Project Summary

Services robots directly interact with the people, so finding a more natural and easy user interface is of fundamental importance. While previous works have been focused
primarily on issues being manipulation and navigation in the environment, some robotics are used with user-friendly interfaces that possess the ability to control the robot by natural means.This project presents the development and testing of a wireless wearable glove to facilitate the accurate measurement of finger movement through the integration of multiple IMU sensors.In our project, these gestures are mapped to robot commands under two different modes of operation: local and global control. In the local control mode, the gestures are interpreted in the robot's local frame of reference, allowing the user to accelerate, decelerate, and turn. In the global control mode, the gestures are interpreted in the world frame, allowing the robot to move to the location at which the user is pointing. The main is to introduce the pointing feature(global control mode) in the gesture control of robot operation so as to avoid stray gestures to get the maximum precision and accuracy by measuring the inclination of the finger joints and translating them to robot operation e.g.direction,speed and distance to travel by the robot with just one single gesture instead of involving the several different gestures which when performed by the users with less technical background are more prone to errors thus providing the ease of use.

Project Objectives

1. Design a wearable glove unit with integrated IMU sensors placed on the palm and finger joints.
2. For the local control gestures derive the inertial motion parameters of a human palm being linear acceleration and angular movement.
3. For the glocal control mode measure the inertial motion parameters of a human finger joint (DIP, PIP, MCP).
4. Derive the orientation and relative orientation of each finger joint.
5. Transmit the local and global control gesture parameters to the host system.
6. Design the mathematical model of the mobile robot and input the inertial parameters from the glove unit.
7. Translates the parameters of motion for the mobile robot for local control gestures.
8. Translates the parameters of motion for the mobile robot for global control gestures.
9. Intergrate viusal feedback with the mobile robot body.

Project Implementation Method

The project consists of a wearable data glove, on which the IMU sensors will be mounted to measure the acceleration, orientation and tilt of the hand which will then be translated to motor movement. To provide the wireless communication with a halfduplex communication channel (HC-12) transceiver module is used as shown in (fig.1)

Figure. 1
The user can either wave in a direction for the robot to move or point at the desired destination in a 'point-and-go' fashion. The system works in either a local control mode or a global control mode. Under the local control mode, the gesture server interprets gestures as if the user is in the robot's local coordinate frame. Under the global control mode, the user and the robot are both working in the universal coordinate frame. The gesture server spots the gesture from the data stream and the interpreter generates the desired velocity vector, or a desired position. They are sent to the robot and converted to linear and angular velocities on-board. The process is shown in figure 2. When the desired position, is sent to the robot, a proportional control algorithm is used to the servo to that position.

Figure.2.

GLOVE DESIGN
For orientation/inclination/acceleration input through the wearable glove, we have tried to design the glove keeping in view the human physiology(restriction/limitation in
movement of wrist and finger joints as shown in fig 2.1.a) in such a way that the sensors could easily measure the motion against the specific gesture. We have mounted three IMU sensors MPU6050 on the index finger for global control operation of the robot and at the palm of the hand, we have mounted one MPU6050 for local control operation of the robot. The fifth sensor is mounted on the ring finger which will
control the speed of robot in local control mode and this sensor also distinguishes between the local and global control modes of operation of the robot.

Figure.2.1.a

The figure 2.1.b shows the complete illustration of glove design where data is being transmitted to robot for further translation into robot movement using a wireless communication channel.

Figure.2.1.b

Benefits of the Project

This project is part of the MEMS Application in Robotics research group based in SSCASE-IT. Our research group focuses on designing and applying MEMS based sensors in the area of robotics to measure and understand inertial parameters which we then use as a testbench to design our own MEMS accelerometer and gyroscope  sensors.

The academic worth of Inertial sensors is dependent on the applications whether its object driving, wearable kits, or motion sensing. In academics, the sensors are in continuous study to improve their parameters and data processing. For motion interface enable devices the key requirements are small package size, low power consumption, high accuracy, high repeatability, high shock tolerance and application specific programmability etc. and the sensor MPU-6050 that we have chosen for our application fulfils all the requirements at a very low consumer price. These sensors can also be used in various industrial applications like in medicine for tracking the exact posture of the human body (posture correction), for sign language interpretation, the movement (e.g. walking or any other movement) of humanoid robots. These sensors also find their use in air vehicles and tanks.
With this project we have aimed to target its applicaltions in biomedical engineering.One of it's best application in regard is Creating gesture controlled games for robot-assisted stroke rehabilitation.Regular training exercises are fundamental to regain functional use of arm and hand control after a stroke. Due to high costs and limited availability of
health care professionals, intensity and/or dosage of rehabilitation is often limited. Hence, any technical device that can prolong neurorehabilitation out of the clinic, with low
cost treatment, plays an important role in the health management systems.With this system, the patient can practice hand excercising independently at home by playing gesture controlled games using a robotic glove(orthosis)for prolonged rehabilitation out of the clinic,with low cost treatment.The system consists of a user interface (UI) on a touch screen, a set of games for training and an orthosis which supports the patient’s movements. The patient’s system is remotely connected to a therapist application for supervision.The orthosis glove basically is a wrist, hand and finger orthosis(external device) that assists individuals after stroke, suffering from impairments caused by spasticity and abnormal synergies. The orthosis offsets these undesired torques with passive springs that pull the joints towards extension. The user carries out voluntary muscle activation to perform movements and thus stays actively involved. The orthosis is equipped with sensors to measure the joint rotations and applied forces at the joints, which are used to interact with a gaming environment. It also provides information on the user's forearm posture and movements.

Technical Details of Final Deliverable

• Design of the data glove and a communication channel with the mobile robot and host system
• How to interface 1 IMU sensor with Arduino
• Extraction of features and use of communication channel
• Addition of multiple sensors
• Interface multiple sensors to microcontroller
• Extraction and Classify features
• Implemntation of local and global gestures
• Visual feedback

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Others

Other Industries

Core Technology

Wearables and Implantables

Other Technologies

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com