FPGA Based Text to Speech System for Paralyse Person

Project Title

FPGA Based Text to Speech System for Paralyse Person

Project Area of Specialization Wearables and ImplantableProject Summary

To lead our life we had to run with our fast materialistic world. To express our thoughts, we communicate with different people in different languages and in different ways. But, it is difficult for the paralyze people who are affected by paralysis to convey what they think and feel extremely difficult to express their ideas. Hence, there is a need to develop a platform for those people who are physically challengeable. Assistive technology can refer to any device that helps a person with hearing loss or a voice, speech, or language disorder to communicate. These terms often refer to devices that help a person to hear and understand what is being said more clearly or to express thoughts more easily.Scientists are close to devising technology that uses the brain's encoding and muscle control commands to allow people who have lost the power of speech due to paralysis to talk again.The finding brings closer the day when people who are paralyzed, such as individuals with "locked-in syndrome" will be able to speak through a "brain-machine interface" by just trying to say words. An Embedded device shall address the above said problems. The impaired person could use some other parts of their body to indicate the desired response. Patients will use their cheek muscles which will be wired to the keyboard button. This method would see the user selecting a symbol or key word and highlighting it at that particular moment. In this project we are using two sensors i.e. one for selecting the desired text based on cheek movement and the other for the conversion of text into speech.

Project Objectives

Communicating effectively is one of the most daunting challenges that paralyzed individuals and their families face. The main objective of this project is to facilitate paralyzed people with easiest communication means. a group of researchers will be able to invent a device that can help these patients to speak by slight movement  of their cheek.

Those suffering with severe forms of paralysis or having significant mobility problems but are mentally healthy to take decisions, this  signal controlled device is designed to  give them the ability to communicate with other. Motion sensors are used to design such a device which will convert text into speech and make communication possible for paralyzed people. This signal controlled device works on a slight movement of cheek muscle with conversion of desired text into speech by using FPGA and motion sensors.

Project Implementation Method

In this project we have two motion sensors which are mounted on glasses. Those sensors are responsible to sense the cheek movement and are interfacing with the LCD to select the text and convert that text into speech. LCD contains the image of virtual keyboard, a combination of pixels. One sensor is responsible to check the cheek movement which is controlled by mini Arduino. On the base of this signal the text is selected and displayed onLCD.

Once we have read the value of sensor by Arduino then we have to send that data to Zybo FPGA part through Bluetooth module (HC05) for selecting the desired text. Zybo architecture has the PL (Programming Logic) part and PS (Processing System part).Bluetooth module is used for wireless communication. We have used one Bluetooth module with Arduino end and another Bluetooth module at Zybo end. The ArduinoBluetooth module is in Slave mode and Zybo Bluetooth module is in Master Mode. These two Bluetooth modules are configured as auto-connect. It will send the data via UART.

We have programmed the LCD by using VGA such that it is divided into three parts. First portion contains the alphabets, Second portion contains the numeric values and special characters; and third portion contains the sentences. We have made a function of toggling which toggle between three parts of LCD. After first triggering the toggling will start between the rows of selected portion. Now user has to move his cheek to select the specific row of selected portion. After second triggering, the specific row is selected and toggling will start column wise which is character by character. Now user needs to move his cheek again to select the desired alphabet. So by repeating this process the person can type a word or sentence. Now if person want to convert this text into speech then he will move the second sensor which will be mounted on another cheek. This movement convert the written text into speech and paralyze person will be able to communicate.

Benefits of the Project

It is not easy for paralyzed people to communicate with others easily. For people with these severe forms of paralysis, a lack of speech can be emotionally devastating .For that purpose we are making a System for paralyzed persons who will detect the motion and type on screen along with conversion of that text into speech. For those suffering with severe forms of paralysis or having significant mobility problems like those associated with quadriplegia, there are a few, not many, available options for communication.. Alternative and augmentative devices allow people to express themselves and to have two-way conversations with caregivers, their loved ones, and others. Assistive technology can help people with varied communication limitations. People with the most severe impairments may be able to speak by using their eyes or another facial feature by directing a device. FPGA based text to speech system will help the paralyze people to communicate.

Technical Details of Final Deliverable

ZYnq Board

The ZYBO (ZYnq Board) is ready to use embedded software and a digital circuit platform builds around the Xilinx family. The Z-7010 is Xilinx All Programmable System on Chip (AP SoC) architecture which integrates a dual core ARM Cortex with FPGA.ZYBO can host a whole system design. Zybo architecture comprises of both the PL (Programming Logic) and PS (Processing System).PL is responsible for all the dealing out of VGA where as PS is responsible for processing of signals through sensors and conversion of typed text into speech.

Motion sensors

The two motion sensors used are PIR (passive infrared sensors) which are mounted on glasses on both of the cheeks through a wire. The term passive itself indicate that sensor doesn’t emit IR signals but passively detects infrared radiations from human body or from the surroundings. Those sensors are connected with arduino and both sensors have different tasks i.e. one gives signal to arduino to select the words from the virtual keyboard and other to sense the cheek movement and give signal to convert the typed text into speech.

LCD

 All the text is typed through a virtual keyboard. The virtual keyboard in our project is designed on a LCD of a resolution of 1280 x 1024 and all the data is sent through VGA. The LCD contains the image of virtual keyboard, a combination of pixels. LCD is programmed in such a way that it is divided into three parts. First portion contains all the alphabets; Second portion contains all the numeric values and the special characters whereas the third portion contains chunk of words. All the words are selected in a row i.e. the cursor moves from left to right on the screen same as cursor moves in a normal word file. This virtual keyboard functions on toggling i.e. toggling of frames over different portion of screen. Frames are programmed and calculated through screen resolution. This toggling of screen is controlled through movement of cheek and signal generated by motion sensor. After first trigger from the sensor, the frames will be converted into row cursor; the toggling will start between the five rows of selected portion reading all alphabets in a rows.

Bluetooth module

Bluetooth module connected with Arduino  and the  other Bluetooth module is connected with Zybo. The module connected with  Ardunio is in Slave mode and the one connected with  Zybo is in Master Mode. Both of  Bluetooth modules are configured on auto-connect and all  data is sent via UART.

Text to speech conversion(Emic2)

To convert the typed text into speech, we are using Emic2.Emic2 is a Multilanguage voice synthesizer which converts a stream of digital text into natural sounding speech which is integrated with embedded system.

Final Deliverable of the Project HW/SW integrated systemType of Industry IT , Medical Technologies Wearables and ImplantablesSustainable Development Goals Good Health and Well-Being for People, Reduced InequalityRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	71160
Zybo	Equipment	1	29500	29500
Pmod BT2	Equipment	2	4234	8467
Blutooth Module (HC-05)	Equipment	2	600	1200
Arduino Mini	Equipment	1	450	450
Arduino Uno	Equipment	1	800	800
Emic2	Equipment	1	13443	13443
Jumper Wires pack	Equipment	2	200	400
LCD	Equipment	1	3500	3500
SD-Card	Equipment	1	600	600
Others	Miscellaneous	7	1400	9800
Sensors	Equipment	2	600	1200
Power Supply	Equipment	1	1800	1800