Handwritten Digit Recognition using Machine and Deep Learning Algorithms

Handwritten digit recognition is the ability of a computer to recognize the human handwritten digits from different sources like images, papers, touch screens, etc, and classify them into 10 predefined classes (0-9). This has been a topic of boundless-research in the field of deep learning. Digit recognition has many applications like number plate recognition, postal mail sorting, bank check processing, etc . In Handwritten digit recognition, everyone faces many challenges because of different styles of writing of different peoples as it is not an Optical character recognition. This project provides a comprehensive comparison between different machine learning and deep learning algorithms for the purpose of handwritten digit recognition while using the Support Vector Machine, Multilayer Perceptron, and Convolutional Neural Network for the same purpose The comparison between these algorithms is carried out on the basis of their accuracy, errors, and testing training time corroborated by plots and charts that have been constructed using matplotlib for visualization. The accuracy of any model is paramount as more accurate models make better decisions. The models with low accuracy are not suitable for real-world applications. Ex- For an automated bank cheque processing system where the system recognizes the amount and date on the check, high accuracy is very critical. If the system incorrectly recognizes a digit, it can lead to major damage which is not desirable. That's why an algorithm with high accuracy is required in these real-world applications. Hence this paper deals with providing a comparison of different algorithms based on their accuracy so that the most accurate algorithm with the least chances of errors can be employed in various applications of handwritten digit recognition.

To compare the algorithms based on working accuracy, execution time, complexity, and the number of epochs (in deep learning algorithms) this project used three different classifiers: Support Vector Machine Classifier, ANN - Multilayer Perceptron Classifier & Convolutional Neural Network Classifier It gives a comprehensive understanding of the implementation of each algorithm explicitly to create a flow of this analysis to create a fluent and accurate comparison.

 Pre-Processing: Pre-processing is an initial step in the machine and deep learning which focuses on improving the input data by reducing unwanted impurities and redundancy. To simplify and break down the input data all the images present have been reshaped in 2-dimensional images i.e (28,28,1). Each pixel value of the images lies between 0 to 255 followed by Normalizing these pixel values by converting the dataset into 'float32' and then dividing by 255.0 so that the input features will range between 0.0 to 1.0. Next, one-hot is performed encoding to convert the y values into zeros and ones, making each number categorical, for example, an output value 4 will be converted into an array of zero and one i.e [0,0,0,0,1,0,0,0,0,0].

 SVM : The SVM in scikit-learn supports both dense (numpy.ndarray and convertible to that by NumPy.asarray) and sparse (any scipy.sparse) sample vectors as input. In sci-kitlearn, SVC, NuSVC & LinearSVC are classes capable of performing multi-class classification on a dataset. This project has used LinearSVC for the classification of MNIST datasets that make use of a Linear kernel implemented with the help of LIBLINEAR . Various sci-kit-learn libraries like NumPy, matplotlib, pandas, Sklearn & seaborn have been used for the implementation purpose. Firstly, download the MNIST datasets, followed by loading it and reading those CSV files using pandas. After this, plotting of some samples as well as converting into matrix followed by normalization and scaling of features have been done. Finally, create a linear SVM model and confusion matrix that is used to measure the accuracy of the model .

 MLP: The implementation of Handwritten digits recognition by Multilayer perceptron  which is also known as feedforward artificial neural network is done with the help of Keras module to create an MLP model of Sequential class and add respective hidden layers with different activation function to take an image of 28x28 pixel size as input. After creating a sequential model, the Dense layer of different specifications and Drop out layers as shown in the image below is added. The block diagram is given here for reference. Once training and test data are available, one can follow these steps to train a neural network in Keras. The project uses a neural network with 4 hidden layers and an output layer with 10 units (i.e. total number of labels). The number of units in the hidden layers is kept to be 512.

To compare the algorithms based on working accuracy, execution time, complexity, and the number of epochs (in deep learning algorithms) this project used three different classifiers: Support Vector Machine Classifier, ANN - Multilayer Perceptron Classifier & Convolutional Neural Network Classifier It gives a comprehensive understanding of the implementation of each algorithm explicitly to create a flow of this analysis to create a fluent and accurate comparison.

 Pre-Processing: Pre-processing is an initial step in the machine and deep learning which focuses on improving the input data by reducing unwanted impurities and redundancy. To simplify and break down the input data all the images present have been reshaped in 2-dimensional images i.e (28,28,1). Each pixel value of the images lies between 0 to 255 followed by Normalizing these pixel values by converting the dataset into 'float32' and then dividing by 255.0 so that the input features will range between 0.0 to 1.0. Next, one-hot is performed encoding to convert the y values into zeros and ones, making each number categorical, for example, an output value 4 will be converted into an array of zero and one i.e [0,0,0,0,1,0,0,0,0,0].

 SVM : The SVM in scikit-learn supports both dense (numpy.ndarray and convertible to that by NumPy.asarray) and sparse (any scipy.sparse) sample vectors as input. In sci-kitlearn, SVC, NuSVC & LinearSVC are classes capable of performing multi-class classification on a dataset. This project has used LinearSVC for the classification of MNIST datasets that make use of a Linear kernel implemented with the help of LIBLINEAR . Various sci-kit-learn libraries like NumPy, matplotlib, pandas, Sklearn & seaborn have been used for the implementation purpose. Firstly, download the MNIST datasets, followed by loading it and reading those CSV files using pandas. After this, plotting of some samples as well as converting into matrix followed by normalization and scaling of features have been done. Finally, create a linear SVM model and confusion matrix that is used to measure the accuracy of the model .

 MLP: The implementation of Handwritten digits recognition by Multilayer perceptron  which is also known as feedforward artificial neural network is done with the help of Keras module to create an MLP model of Sequential class and add respective hidden layers with different activation function to take an image of 28x28 pixel size as input. After creating a sequential model, the Dense layer of different specifications and Drop out layers as shown in the image below is added. The block diagram is given here for reference. Once training and test data are available, one can follow these steps to train a neural network in Keras. The project uses a neural network with 4 hidden layers and an output layer with 10 units (i.e. total number of labels). The number of units in the hidden layers is kept to be 512.