Enhancing Feed-Forward Neural Network in Image Classification

In this paper. the researcher use Feed Forward Neural Network for image classification. The objective of this paper is to Enhancing the structure of FFNN by adding dropout method into the input layer and hidden layer to prevent over-fitting the network and then initialized by random uniform initializer for each layer then setting a bias of zeros, also adding the following activation function ReLu&Softmax to the network and the dataset that the researcher use was the MNIST-handwritten numbers. The dropout method is one of the easy and efficient regularization methods that avoids overfitting which stochastically drops out some neurons and trains the network at each weight update. After that, researcher determines the performance of the result by evaluating the FFNN with dropout on the basis of loss and accuracy to take the average of the difference between the standard FFNN and CNN. The result of FFNN with dropout method had an average of 99.86% accuracy and 0.47% of loss, the standard FFNN got 98.13% accuracy and loss of 9.15%, while the CNN had an average of 99.26% accuracy and 2.39% of loss. As the training process of neural network is an iterative process comprising forward propagation, speed improvement using dropout would also provide a significantly decreased training time.