Training feedforward neural network for OCR

Examine this example program Handwritten Digit Recognation

Program uses a Semeion Handwritten Digit Data Set with FANN library

You probably want to follow Lectures 3 and 4 at http://www.ml-class.org. Professor Ng has solved this exact problem. He is classifying 10 digits (0...9). Some of the things that he did in the class that gets him to a 95% training accuracy are :

• Input Nueron : 400 (20x20)
  • Hidden Layers : 2
  • Size of hidden layers : 25
  • Activation function : sigmoid
  • Training method : gradient descent
  • Data size : 5000

For handwritten character recognition you need

1. many training examples (maybe you should create distortions of your training set)
2. softmax activation function in the output layer
3. cross entropy error function
4. training with stochastic gradient descent
5. a bias in each layer

A good test problem is the handwritten digit data set MNIST. Here are papers that successfully applied neural networks on this data set:

Y. LeCun, L. Bottou, Y. Bengio and P. Haffner: Gradient-Based Learning Applied to Document Recognition, http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf

Dan Claudiu Ciresan, Ueli Meier, Luca Maria Gambardella, Juergen Schmidhuber: Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition, http://arxiv.org/abs/1003.0358

I trained an MLP with 784-200-50-10 architecture and got >96% accuracy on the test set.