What does it mean when train and validation loss diverge from epoch 1?
Possible explanations
- Coding error
- Overfitting due to differences in the training / validation data
- Skewed classes (and differences in the training / validation data)
Things I would try
- Swapping the training and the validation set. Does the problem still occur?
- Plot the curves in more detail for the first ~10 epochs (e.g. directly after initialization; each few training iterations, not only per epoch). Do you still start at > 75%? Then your classes might be skewed and you might also want to check if your training-validation split is stratified.
Code
- This is useless:
np.concatenate(X_train)
- Make your code as readable as possible when you post it here. This includes removing lines which are commented out.
This looks suspicious for a coding error to me:
if test:
X_test.append(input)
y_test.append(output)
else:
#if((output == 0 and np.average(y_train) > 0.5) or output == 1):
X_train.append(input)
y_train.append(output)
use sklearn.model_selection.train_test_split
instead. Do all transformations to the data before, then make the split with this method.