IoT based Electromyography and speech controlled Robotic Car using Raspberry Pi
Wearable electronic equipment is continually improving and becoming more integrated with technology for prosthesis control. These devices, which come in a variety of shapes and sizes, sense and measure physiological and muscular changes in the human body and may then utilize those signals to operate
2025-06-28 16:28:04 - Adil Khan
IoT based Electromyography and speech controlled Robotic Car using Raspberry Pi
Project Area of Specialization Electrical/Electronic EngineeringProject SummaryWearable electronic equipment is continually improving and becoming more integrated with technology for prosthesis control. These devices, which come in a variety of shapes and sizes, sense and measure physiological and muscular changes in the human body and may then utilize those signals to operate machines. One such gadget, the MYO gesture band, gathers Electromyography (EMG) data utilizing myoelectric impulses and converts them to input signals using pre-defined motions. Use of this device in a multi-modal environment will not only increase the possible types of work that can be accomplished with the help of such device, but it will also help in improving the accuracy of the tasks performed. This project addresses the fusion of input modalities such as speech and myoelectric signals captured through a microphone and MYO band, respectively, to control a robotic car.
This project is completed in three stages. A comprehensive study of technologies based on the multimodal environment was done during first phase. The purpose was to determine the benefits and drawbacks of introducing multimodality in the project. Traditionally, the robotic is controlled by unimodal systems such as either speech, EMG, EEG etc. However, in this project we have proposed multimodal system, which includes multiple input methods such as speech and EMG to control the robotic accurately. In the literature, EMG data has not been employed in many multimodal systems, and when it has been used, it has resulted in a high accuracy fusion that is effective for real-world application.
In the second phase of the project, prototype is designed and developed. The robotic car is controlled by speech and EMG data. The speech signal is processed using a speech recognition google API whereas the EMG is acquired using the MYO armband.
The results obtained in phase 2 are improved in the last phase of the project by enhancing the Feature Extraction and Classification to acquire the best approach for accurately controlling a robotic car.
Project Objectives- To learn about EMG and its types
- To learn how to get speech data
- EMG data Acquisition using MYO armband
- To understand working and operation of Raspberry Pi
- Programming in Pi
- To learn Interfacing the microphone with robotic car
- To learn Interfacing the MYO armband with robotic car
- To apply feature extraction methods
- To apply classification method
- Offline analysis of classification algorithm on EMG data
- To design a prototype robotic car that can be controlled by human-speech and EMG MYO armband.
Initially, we control our robotic car via MYO armband, and later voice control was incorporated. Both are explained one by one below:
The robotic car is controlled by an electromyography (EMG) data-based armband called a MYO armband. The armband can capture five gestures: fist, wave left, wave right, double tap, and fingers spread. The Raspberry Pi 3 is featured in the robotic car to take input from MYO armband through USB.
Robotic car has four Geared Motors, and the Raspberry Pi is connected to the car with pins specified for each motor in robotic car. When the user performs gestures using his/her arm, input is transmitted from the MYO armband to the Pi, and as a result, it moves the specified motor. Usually, the collected sEMG signals are normally noisy due to ambient noises and inherent instability of the sEMG signal. So, Feature extraction in time-domain is performed before utilizing the signal. The value of this feature extraction will be the input for the classification model. The time-domain features extracted here are Root mean square, Mean, Variance, and Kurtosis. We used a sliding window of step 5s for each sensor. So, for 1000 samples, we got 200 data points for each hand movement, which are then fed into an algorithm, named as SVM (Support Vector Machine) for classification.
There are two models of the SVM classifier, linear and the RBF (Gaussian). The RBF kernel of the SVM classifier is adopted in our project. The iterative method grid search is used to achieve optimum values, where the optimal hyperparameters output of grid search method are C = 1 and gamma = 0.
For voice control we are using an externally connected USB microphone with raspberry pi to take input voice command. Voice command, using microphone is sent to the google API and google API analyzes the command and converts it to the text and then the text will be sent again to Raspberry Pi. Any spoken word / phrase (Voice Commands) are converted to text and passed to the robot. However, valid voice commands (stop, forward, backward, left, and right) will be followed by the action that needs to be performed. To read the text, we created a python code and after reading the text we are able to output voltages to the 4 pins on the car.
The inclusion of speech input modality improved the accuracy of the MYO band significantly. While using the modalities separately, the accuracy was 86.54%9 and 82.06% for the MYO and Speech. After fusion of the inputs, accuracy improved to more than 95.92%.
Benefits of the Project- It will be easily adaptable for amputees, paralyzed people
- A new research horizon will open up. (Improvement, fusion with other devices etc.)
- It is a low-cost project
- Multimodal systems can offer a flexible, efficient and usable environment allowing users to interact through input modalities, such as speech, hand gestures etc.
Final Deliverable will be a human-speech and EMG MYO armband controlled prototype robotic car.
MYO Armband will be used for gestures control and EMG signals interfacing
Microphone will be used for capturing voice signals and after Analogue/Digital and Digital/Analogue conversion car will be controlled by voice commands given by user in microphone.
Raspberry Pi will be used as microcontroller and all signal processing will be done by it.
H-bridge will control the direction and speed of motors
Final Deliverable of the Project HW/SW integrated systemCore Industry ManufacturingOther IndustriesCore Technology RoboticsOther TechnologiesSustainable Development Goals Decent Work and Economic Growth, Industry, Innovation and InfrastructureRequired Resources| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Total in (Rs) | 70000 | |||
| MYO Armband | Equipment | 1 | 49000 | 49000 |
| Raspberry Pi 3 with Power Supply | Equipment | 1 | 16000 | 16000 |
| Robotic car | Equipment | 1 | 2000 | 2000 |
| Electronic components | Equipment | 1 | 3000 | 3000 |