How can I add noise to a neural network input?
For what it's worth, this is my "add binomial random variate layer":
Needs["NeuralNetworks`"] (* for ScalarTimesLayer, ScalarPlusLayer *)
With[{σ = 0.1, n = 5},
addNoiseLayer = NetGraph[
Flatten@{
TotalLayer[], ScalarTimesLayer[σ*1./5],
ScalarPlusLayer[-1*σ], TotalLayer[],
Table[DropoutLayer[], {n}]
},
Flatten@{
1 -> 2 -> 3 -> 4,
NetPort["Input"] -> 4,
Table[NetPort["const"] -> 4 + i -> 1, {i, n}]
}, "Input" -> inputDim, "const" -> inputDim]]
Use NeuralNetworks`NoiseLayer
<<NeuralNetworks`
noisy = NetGraph[{
"noise" -> NoiseLayer[{3, 90, 160}, "Distribution" -> NormalDistribution[0, 0.1]],
"add" -> ThreadingLayer[Plus]
},
{{"noise", NetPort["Input"]} -> "add"},
"Input" -> NetEncoder[{"Image", {160, 90}}],
"Output" -> NetDecoder["Image"]
]
You can find more undocumented definitions with
NeuralNetworks`$LayerData["Noise"]//Dataset
In Mathematica 12.1
, DropoutLayer exposes his mask, so it's possible to train using a random binary mask, with DropoutLayer["OutputPorts" -> "BinaryMask"]
(or DropoutLayer["OutputPorts" -> {"Output", "BinaryMask"}]
)
In Mathematica 12.2
, it will be possible to simulate many types of noises (including once where you give the parameters -like mean and variance- through outputs of other subparts of a NetGraph).
The novelty is already documented in the current 12.2 builds under the name of RandomArrayLayer
.