Using Gekko's brain module, how do I determine how many layers and what type of layer to use to solve a deep learning problem?

I get the following result if i increase the nodes to 5 b.layer(tanh=5)

There are probably multiple answers to this question, tho. Maybe increasing the number of layers or changing the activation function. You can always use different solvers, too. Finding the best network architecture is an optimization problem of its own. Some people have tried to figure it out with genetic algorithms, for example:

https://arxiv.org/pdf/1808.03818.pdf

Data-driven (Black-box) models inherently have "Overfitting" and "Underfitting" problems. If you give too many degrees of freedom in your model, it will perfectly fit into your "Training" data set, while it won't fit that well with the "Validation" data set.

Same for the Neural Net type of models. The more layers you give, the easier the model results in the "Overfitting."

There are several ways to avoid "Overfitting".

Generally, you can balance the "model errors" between the "Training set" and "Validation set," stopping increasing the layers at the point when the Validation Error starts increasing while the training error keeps decreasing, or vice versa.