OPTIMIZATION AND REAL-TIME ANALYSIS OF BIG TRANSACTIONAL DATA

Project Title

OPTIMIZATION AND REAL-TIME ANALYSIS OF BIG TRANSACTIONAL DATA

Project Area of Specialization Computer ScienceProject Summary

Our project is to explore and analyze Big data to optimize and create a benchmark of it for future use. In this project, we are going to use different tools and software programs to run queries on Big Data and analyze whether our system crashes or not. We are going to configure our systems in order to run queries on Big Data. In order to do so, we are going to divide this Big Data into Clusters. When the query runs on a system configuration successfully with a specific tool, without crashing, we will move on to a new tool. When all the tools are exhausted, we will increase the size of these clusters of our big data and then perform different queries with the same tools on those clusters. We will be doing so until we have a stopping point i.e. the system crash. By increasing the size of the cluster after running queries using all the tools, we will have to determine which tools to use on which size of cluster using which system in order to ensure that our system is optimized as well as up and running throughout our query process. We will also monitor and benchmark the data, which will be used to generate a dashboard in real-time.

Project Objectives

Big Data introduces new schemes, enabling us to know customers’ profiles, soothsaying their behavior, and evaluate risks. The objective of our project is to optimize big transactional data in order to amend operations of the systems, provide better customer accommodation, engender personalized marketing campaigns, and take other actions that ultimately increment revenue and profits. This will allow the user or organization to measure their performance metrics and compare them to improve their results. By doing the real-time analysis of the transactions, the companies can generate a dashboard indicating the statistical analysis of the transactions, preserving the time of the customer as well as of the company, by fetching the data from the root or looking into each and every file for a particular record.

Project Implementation Method

Handling big transactional data is extremely difficult. Big transactional data is mainly used by banks. Hence it cannot be managed on personal computer systems. Proper configuration of systems and precise division of data in clusters can help manage the problems of big data. Our requirement is to optimize bank transactions through big data queries, then design a real-time dashboard and implement queries into it. We will be following the Waterfall methodology to support our progress. Big data tools to be used are: • Apache Hadoop • Apache Spark • Apache Hive • Apache Tez • Apache Kylin 

Use cases will be tested accordingly.

Benefits of the Project

The motivation for opting for this project is foremost a requirement of today's technological world. This demanding project is a golden opportunity to acquire knowledge about a globally increasing problem of big data. Big Data is one of the biggest problems for most organizations of today. An immense amount of data is generated every second, from business transactions to customer logs. All of this data is being piled up into worthless bulk so it is high time that it gets optimized. So, it can be used in the most efficient manner. Moreover, real-time analytics enables immediate action, allowing businesses to be motivated and proactive.

Technical Details of Final Deliverable

The expected outcome of our project is the best possible way of Optimizing Big Data. And also presents a real-time analytic dashboard of that data.

Final Deliverable of the Project HW/SW integrated systemCore Industry ITOther Industries Telecommunication Core Technology Big DataOther TechnologiesSustainable Development Goals Industry, Innovation and InfrastructureRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	80000
Server PC	Equipment	1	70000	70000
Accessories	Miscellaneous	1	10000	10000