WordsInAction

Project Title

WordsInAction

Project Area of Specialization Artificial IntelligenceProject Summary

If a picture tells a thousand words than animation tells a million, nowadays animation is the most effective and attractive way to make anything understandable, to someone, and to entertain. Also, visual modalities are one of the most important modalities in any
multimedia presentation. Making someone understand a scenario or to give a group of people the same view through language is quite a difficult task because the way of thinking varies from person to person. So each individual would think differently. And here animation plays an important role apart from entertaining.

Moreover, in recent years, due to the digital revolution the paradigm of the world is shifting towards automation and everything from watches to complex machines is being automated. But nowhere have we seen the automation of the animation which is quite an interesting and significant thing for the society which is preferring animation in every prospect of life.

So, the core idea of the project, “Words in Action” is to automatically convert the natural
language into animation. A desktop application that would help people to visualize the
written scripts and scenarios to have a clear image of what is written.

Project Objectives NLP OBJECTIVES Replacing Pronouns:

Every story contains pronouns that represent a specific noun. To extract the exact information about the character we need to have proper nouns instead of a pronoun. So the first objective is to replace the pronouns with nouns.

Extracting information of each character:

To extract the information. We first need to parse into the sentences, the whole story. Now by clustering all the sentences from the story on the character name basis, we can have information of all characters separately.

Action Identification and Extraction:

Now having all the information of each character we can identify the action each character is performing and the sequence of there action

ANIMATIONS OBJECTIVES Action Mapping:

Having all the actions of respective characters, now the objective is to map the action of each character of the story to the animator (a character in animated video). Keeping in view the sequence of all the action, each character in the animated video is assigned a sequence of animation according to the action 

Camera Handling:

One of the critical objectives is to make that character view to the user who performs any task. As we have only one screen, so here scheduling is needed to make any character view on screen

Model (RESNET100):

It would be used to classify which animation is to perform against which action. As there are multiple classes of animation as a dataset

Project Implementation Method NLP IMPLEMENTATION Replacing Pronouns:

Replacing pronoun with there noun was the critical most task. For that stanford corenlp that is the pre-trained model and is used to map the pronoun to its noun. After using that we get a JSON object on which we apply couple of algorithms to get the required output 

Extracting information of each character:

For this POS tagger is used. On every parsed sentence we apply POS tagger that tag every word of the sentence. Given the tag, we check the property of the word and on the basis of these actions are extracted.

Action Identification and Extraction:

Now having all the information of each character we can identify the action each character is performing and the sequence of there action

ANIMATIONS OBJECTIVES Action Mapping:

resnet model is used to map the action of the character to the animation form the data set

Camera Handling:

There are multiple cameras taking a shot at the same time at different places. That camera is enabled under which any character is performing any task.

Benefits of the Project

Automatically generating the animation from the natural language is an interesting and
new field. And so has benefits for an enormous market.
This project deals with making the animation of the house robbery scene so the system would be able to generate the animation against the eyewitnesses testimonies. That will help in visualizing the scenario to the investigator. All the investigators would be on the same page once a perfect 3D animated video is generated. An effort was made to generate the scene on virtual reality but nowhere in 3D animation. Furthermore, customers can be news channels. Every news channel always tries to attract more and more people with different strategies. And now they always try to show an animation of the news which they are unable to capture like a robbery scene. So, nowadays many of the news channels are shifting towards animation. And why don’t they, when everyone prefers animation overwritten scripts and even reading them. Albeit, making animation is a long process and so is an issue for the news channels because they have to cast the news at their earliest. So, this project would provide animation at the very next moment as the user enters the scenario. And would not only add colors to their news but also make the audience more conscious and careful about these incidents as they would be watching what had happened. Moreover, these types of
projects always play an important role in reducing expenses.
Other than that, this project could be used by any student to test one creativity, having a scenario of robbery one has to write a story.

Technical Details of Final Deliverable

Finally, we have to integrate the NLP work with that of animations.  For that, a bridge is to be made to make both of the parts communicate with each other. The processed output of NLP work would be the input of unity that will be of assigning the tasks, direction, and action to the number of characters the story has.

Final Deliverable of the Project Software SystemCore Industry EducationOther Industries IT Core Technology Artificial Intelligence(AI)Other Technologies 3D/4D Printing, Cloud InfrastructureSustainable Development Goals Industry, Innovation and Infrastructure, Partnerships to achieve the GoalRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	40000
ZT-T16600K-10M GPU	Equipment	1	40000	40000