Haptic and Forced Feedback Enabled by Exoskeleton Based Glove for VR Applications

Virtual Reality has been focus of researchers from various fields of study, specially since last decade a lot of work has been done to make the immersive world as real as possible. This is achieved by creating devices that give haptic and forced feedback to users in VR environment. Previously done w

Project Title

Haptic and Forced Feedback Enabled by Exoskeleton Based Glove for VR Applications

Project Area of Specialization

Wearables and Implantable

Project Summary

Virtual Reality has been focus of researchers from various fields of study, specially since last decade a lot of work has been done to make the immersive world as real as possible. This is achieved by creating devices that give haptic and forced feedback to users in VR environment. Previously done work in this area usually includes solutions that target either the haptic feedback or forced feedback. Some final products for attaining the said task are present in market but they cost a lot. We propose a cost-effective exoskeleton based glove that will provide both feedbacks that enrich the experience of VR applications. The dimensions of glove will be balanced with optimal layout. It will have vibration motors to provide haptic feedback, that will help user to feel and determine the texture of object. The exoskeleton will help mimic the force that is exerted by objects on human fingers while grasping them. The exoskeleton will be developed using 3D printing and will be drived with aid of servo motors. It might also have other haptic feedback devices such as temperature. Interaction between user and VR environment (created with Unity 2019) will be done using Oculus Go, MS Kinect, that will track skeletal movements, and Leap Motion Sensor/ Flex Sensors, that will track movements of hands. The communication between VR environment and glove will be done wirelessly using Esp8266 NODE MCU. Upon interaction with certain virtual object, Unity will send pre-coded command to Arduino Mega2560 R3 via Esp8266 NODEMCU resulting in driving of vibration motors, exoskeleton, and TEC1-12715 Thermoelectric Cooler Peltier, thereby producing haptic and forced feedback for user.

Block diagram of proposed system is given in figure 1:

figure 1: Conceptual Design of System (images/Haptic and Forced Feedback Enabled by Exoskeleton Based Glove for VR Applications _1639951108.png)

Project Objectives

Our aim is to make a cost-effective exoskeleton based glove that will provide haptic and forced feedback to our hand for enriching the experience of Virtual Reality and removing the boundaries between real and virtual object.

The key objectives for achieving our aim are stated under:

• To develop a VR environment in Unity 3D, for Oculus GO, and implementing physical models on it, such as colliders.
• To design and fabricate the glove with flex sensors, vibration motors, heating/cooling module, and exoskeleton.
• To read hand movements using Flex Sensors or Leap Motion Controller, also to use Kinect for complete skeletal tracking.
• To interface the fabricated glove with VR environment using UDP (wireless communication).
• To interface sensors and actuators with Arduino.
• To achieve sense of touching/grasping different virtual objects with help of actuators using variations in frequency of their driving mechanism.
• To use Oculus Go for better stereo vision and immersive experience, wirelessly.

Project Implementation Method

As our project is based on Virtual Reality, for that we will create a virtual environment in Unity 2019. Then with the help of Kinect device and Flex sensors skeletal tracking of user will be done and engrossed in designed virtual environment. The fabricated glove and virtual environment will be bridged with help of UDP communication using Wi-Fi module. Now whenever the user will try to interact with virtual objects, the data from skeletal trackers and collision detectors will be processed by Unity. It will then send the corresponding triggering commands to Arduino for activating the actuators. By variating operating frequencies of vibration motors on each finger we can provide touch sensations of various textures, such as soft/hard. The exoskeleton drived with aid of servo/DC motors will restrict the movements of fingers when we will grasp an virtual object accordingly. Last but not the least the TEC1-12715  thermoelectric Cooler Peltier will provide sensation of hotness of object whenever we touch/grasp it with respect to its input voltage.

The flow chart for implementation method is given in figure 2.

figure 2: Flow Chart (images/Haptic and Forced Feedback Enabled by Exoskeleton Based Glove for VR Applications _1639951110.png)

Benefits of the Project

In the age of advancing technologies virtual reality is taking over many conventional ways of doing things. Should it be gaming for passing leisure time, training a soldier for battle of survival, or giving therapy exercises to patients, we will find application of VR everywhere around us. The experience of this immersive world can be enriched and more realized with the help of haptic and forced feedback systems. Following are benefits that we will get from the cost-effective feedback glove:

Gaming: A new horizon of gaming will open, where one can not only control the avatar using his real-time movements but also can feel it.

Surgical Training/Practicing: Using simulation environments a surgeon can practice his next major surgery and see how things will go on aforehand. Patients medical condition can be fed to simulator and in virtual environment, with realistic feedback a surgeon can carry on operation and try various possible methods to find the best one. This also might help medical students to try what they have learned in an immersive world.

Physical Therapy: With help of VR glove a physiotherapist can monitor the movement patterns of patient’s hand and make him engaged in interactive virtual exercises, even when they are miles away from each other. The forced feedback exoskeleton in this case can also be used as device for therapy of impaired limbs.

Industries: In industries the lack of working experience with heavy machinery can lead to dangerous accidents, which might go from the scale of impairment of a limb to fatal. With help of interactive simulators one can train how to operate a machine and polish his skills.

Technical Details of Final Deliverable

The final deliverable will be an exoskeleton based glove with haptic and forced feedback systems that involve vibration motors, TEC1-12715 Thermoelectric Cooler Peltier, Servo/DC motors (for exoskeleton) operated by Arduino Mega2560 connected via ESP8266 NODEMCE, that also bridges the glove and virtual environment. The glove will accommodate five flex sensors, five vibration motors, one TEC1-12715 Thermoelectric Cooler Peltier, five servo/DC motors, and an exoskeleton. The VR environment will be developed for Oculus GO to provide stereo vision wirelessly, and for real-time skeletal tracking of user Kinect device will be used.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

IT

Other Industries

Core Technology

Wearables and Implantables

Other Technologies

Internet of Things (IoT), Augmented & Virtual Reality, 3D/4D Printing

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com