Batch normalization during testing

A record of the empirical mean and variance is taken at training time, such as a running average, which is later used for the test set, instead of calculating the means and variances for each test batch.

When you are predicting on test, you always use train's statistics - be it simple transformation or batch normalization.

I'd recommend trying cs231n course to know more about this. Here is how I coded batch normalization while doing this code: github link.

If test statistics significantly differ from train, this means that test is different in general and the model won't work well. In this case you'll need to find different training data anyway. But to be more precise - when you train model on data, processed in a certain way, it won't work well on data, which is processed in a different way.

Let's imagine that there is only 1 test sample - i. e. you want to make a prediction for one client or whatever. You simply can't calculate test statistics in this case. Secondly, let's take batch normalization. Data is normalized and values now show by how many standard deviations original data differes from a certain average. So the model will use this information for training). If you normalize test data using test statistics, then values will show deviation from a different average.