Leg Amputee Automated Prosthesis

Project Title

Leg Amputee Automated Prosthesis

Project Area of Specialization Artificial IntelligenceProject Summary

This summary is about the project, named LEAP (Leg Amputee Automated Prosthesis), that involved the analysis, design and development of a smart and intelligent controller for a lower limb prosthesis, previously designed for a transfemoral amputee. The primary aim of that controller is to provide a more natural, symmetric and synchronized motion with both of the user's intact leg and the prosthetic device.

Last year, students of Robotics and Intelligent Systems Engineering (RISE) lab at School of Mechanical and Manufacturing devised a mechanical prosthesis for a subject with a transfemoral amputation. This prothesis utilizes a simple control mechanism, whereby the speed of the shank of the user is used to vary the stiffness of the knee joint employed in the prosthesis, with an effort to regulate the knee joint as the person starts to walk. We plan to extend the work on this prosthesis by implementing our controller on it, enabling it to first recognize the user's intention of motion (walking, ramp ascent, ramp descent) and using that information, in real-time, to vary the impedance of the knee joint, to provide a more natural gait to the user, while also keeping a balance between the cost and functionality.

The design involves a double layer controller; a high-level controller and a low-level controller. The main purpose of the high-level controller is to use its algorithm to predict the user's locomotion mode intention, just about when he or she is about to perform it, and pass the signal to the low-level controller. The low-level controller will process that signal to set the mechanical impedance of the prosthesis accordingly.
The work on the project has officially started in September 2019 with literature review and meetings with the professionals in the field and is expected to conclude in April 2020 after rigorous testing and debugging. The total budget of the project is Rs. 40,000.

Project Objectives

The controller would enable its users to perform daily routines with increased functionality and the feeling of a sound living limb. The objectives of this FYP are summarised in the following list:

• Literature review on the conventional control techniques for lower limb prosthesis
• Comprehensive study for gait analysis of healthy and unilateral amputee subjects
• Study of the design of the available semi-active lower limb prosthesis
• Generate dataset from healthy subjects to train neural networks or SVMs for locomotion mode intention recognition
• Deploy the trained algorithm on the hardware for real-time predictions
• Set the mechanical impedance of the prosthesis based on the predictions for smooth regulation of knee joint
• Provide a feel of sound limb with a more natural, symmetric gait with increased comfort and functionality
• Deploy the multi-layer controller on a lower limb prosthesis for verbal approval of the candidate upon increased functionality and comfort
• Preliminary experimentation and test results

Project Implementation Method

Our controlled was designed using a multi-layer approach i.e. a high-level controller and a low-level controller. The high level controller involved deploying body-worn sensors to collect the kinematic activity data in real-time and send it to our high-level controller i.e. Raspberry Pi. Appropriate deep learning algorithm e.g. Convolutional Neural Network, was pre-trained using datasets from Kaggle as well as the dataset generated using the body worn sensors. The trained algorithm was deployed onto the Raspberry Pi, for real-time predictions on unseen data from the body worn sensors. This unseen data will be taken from the sensors deployed on the intact leg and sent wirelessly to the Raspberry Pi.

The Raspberry Pi, on the intact leg, will run the deep learning algorithm and predict whether the user is walking on a level ground, up a ramp or down a ramp. Based on this prediction, it sends a regulation signal to the impedance controller which is deployed onto the prosthesis itself. This signal is used by the finite-state machine to automatically set the mechanical impedance of the knee joint of the prosthesis, thus ensuring smooth movement of the prosthesis and in synchronization with the intact leg.

Benefits of the Project

The benefits of this project can be summarized below:

• Adds the functionality of level ground walking, ramp ascent and ramp descent to an otherwise mechanical prosthesis capable of only allowing level ground walking to its user
• A cheaper, yet novel solution, not yet available in the context of Pakistan. Similar solutions cost thousands of USD in Europe and USA. Hence, an effective cost-functionality balance is achieved through this solution
• A controller that automates a mechanical prosthesis to function smartly, providing better user experience and conformity
• The project enables its users to achieve the feeling and functionality of a sound living limb with a more natural and symmetric gait.
• This solution helps its users to be more self-confident, reducing their sense of dependence on others and enhancing their quality of life

Technical Details of Final Deliverable

We incorporated three inertial measurement units (IMUs) Pololu MinIMU-9 v5; one on the calf, one on the thigh and one on the ankle to calculate the roll, pitch and yaw values i.e. the rotation rates around the three (x, y and z) axis of the corresponding parts of the lower limb. The information from all three IMUs will be collected at multiplexor MUX TCA9548A and simultaneously sent through i2c communication to the slave controller of the high-level controller i.e. the Arduino UNO. From there it will be wirelessly transmitted using wireless module NRF240L01 to the master controller of the high-level controller i.e. Raspberry Pi 3B placed on the prosthesis. At the master controller, data logging will be performed in real time and the predictive algorithm i.e. convolutional neural network (CNN) will be executed.

This algorithm is pre-trained using Kaggle datasets of gait activity and is used to output a prediction based on the kinematic data being input via the IMUs. The pre-training was performed on Google Colab Notebook using Tensorflow and Keras as the deep learning frameworks and Python as the language for coding. The computations are performed on a Tensor Processing Unit (TPU) on the cloud. The trained algorithm is deployed onto the Raspberry Pi using Tensorflow Lite. Algorithms like this take useful features as inputs, and through different strategies e.g. Stochastic Gradient Descent or Back Propagation, try to reduce the error between their predicted outcomes and the actual outcomes and thus learn from the dataset in hand. The outputs from these networks are the predictions (whether the user is walking on a level ground, ascended ramp or descended ramp) that the software makes, in real-time, about the intent or activity of the user wearing the prosthesis. These predictions can then further be used by the hardware interface to perform critical control actions based on the variable stiffness of the spring-damper based prosthesis.

The final output from the high-level controller after computing the predictive algorithm i.e. the user predicted intension is input to the finite state machine which is implemented on the Arduino Uno microcontroller, deployed on the prosthesis itself.  The finite state machine has pre-set values of the mechanical impedance according to the different locomotion mode saved. This machine decides the impedance for the prosthesis and informs the impedance controller, which regulates the stiffness of the spring-damper system of the knee joint for smooth, real-time regulation.

Final Deliverable of the Project Hardware SystemCore Industry HealthOther Industries IT Core Technology Artificial Intelligence(AI)Other Technologies RoboticsSustainable Development Goals Good Health and Well-Being for PeopleRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	41955
Raspberry Pi 3B	Equipment	2	6500	13000
Arduino UNO	Equipment	3	700	2100
Multiplexor TCA9548A	Equipment	2	450	900
Pololu MiniIMU-9 v5	Equipment	6	2100	12600
Jumper Wires	Equipment	5	100	500
Wireless Module NRF240L01	Equipment	4	600	2400
Hard Colored Poster Printing	Miscellaneous	4	120	480
Colored Poster Printing	Miscellaneous	10	20	200
B&W Poster Printing	Miscellaneous	25	3	75
Standee Poster	Miscellaneous	2	1000	2000
Transportation/Delivery	Miscellaneous	4	1000	4000
Breadboard	Equipment	2	150	300
Force Sensitive Resistors (FSRs)	Equipment	4	850	3400