How to change NeuralNetwork options?

As already mentioned, options are accessed via:

Method -> {"NeuralNetwork", "L2Regularization" -> 0.12,  "HiddenLayers" -> {4, 3, 3}}

To manually set the activation functions of the first and second hidden layers:

"HiddenLayers"-> {{4, "RectifiedLinear"}, {3, "Tanh"}, 3}

Currently, the following activation functions are supported:

{"LogisticSigmoid", "RectifiedLinear", "Tanh", "SoftRectifiedLinear", "Linear"}

Non-zero "L1Regularization" is not yet supported.

The usual proviso: These options are undocumented functionality whose form may change substantially in future versions.

At least

Method -> {"NeuralNetwork", "L2Regularization" -> 0.01,  "HiddenLayers" -> {4, 3, 3}}

seems to work. "L1Regularization" doesn't seem to be settable. Haven't found out about activation functions yet.

Update

More options can be found using

Options@MachineLearning`PackageScope`NeuralNetworkPredictor 

{"BinaryEncoder" -> "Identity", "ClassNumber" -> Automatic, "CrossValidation" -> Automatic, "DataSize" -> All, "EarlyStopping" -> Automatic, "HiddenLayers" -> Automatic, "L2Regularization" -> Automatic, "LearningTask" -> "Predict", MaxIterations -> Automatic, NominalVariables -> Automatic, PerformanceGoal -> Automatic, "PrintCost" -> False, "OptimizationMethod" -> Automatic, "WeightInitializationMethod" -> Automatic}