Options for ValidationSet accuracy from NetTrain?
I think for the ValidationSet -> Scaled[.1]
the validation will be sampled with randomly from the training data. If we want to control the sample ourselves, we can split them manually and supply it as ValidationSet -> testData
.
We can use the TrainingProgressFunction
function to get more information about the validation accuracy, and abort the training after a certain goal is achieved.
For example, this will stop the training process once the validation loss is smaller than 0.1:
resource = ResourceObject["MNIST"];
trainingData =
RandomSample[ResourceData[resource, "TrainingData"], 6000];
testData = RandomSample[ResourceData[resource, "TestData"], 1000];
lenet = NetChain[{
ConvolutionLayer[20, 5], Ramp, PoolingLayer[2, 2],
ConvolutionLayer[50, 5], Ramp, PoolingLayer[2, 2],
FlattenLayer[], 500, Ramp, 10, SoftmaxLayer[]},
"Output" -> NetDecoder[{"Class", Range[0, 9]}],
"Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale"}]
];
loss = {};
trained =
NetTrain[lenet, trainingData, ValidationSet -> testData,
MaxTrainingRounds -> 20,
TrainingProgressFunction -> {AppendTo[loss, #ValidationLoss];
If[#ValidationLoss < 0.1, Abort[]]; &,
"Interval" -> Quantity[1, "Rounds"]}]
loss
(* {None, 0.474543, 0.290683, 0.174869, 0.146322, 0.110496, 0.109694, 0.0923467} *)
I'll add to our ToDo list the idea of having the validation indices be available as a property of NetTrain.