Difference between Dense(2) and Dense(1) as the final layer of a binary classification CNN?
This first one is the correct solution:
keras.layers.Dense(2, activation = 'softmax')(previousLayer)
Usually, we use the softmax
activation function to do classification tasks, and the output width will be the number of the categories. This means that if you want to classify one object into three categories with the labels A
,B
, or C
, you would need to make the Dense
layer generate an output with a shape of (None, 3)
. Then you can use the cross_entropy
loss function to calculate the LOSS
, automatically calculate the gradient, and do the back-propagation process.
If you want to only generate one value with the Dense
layer, that means you get a tensor with a shape of (None, 1)
- so it produces a single numeric value, like a regression
task. You are using the value of the output to represent the category. The answer is correct, but does not perform like the general solution of the classification
task.
The difference is if the class probabilities are independent of each other (multi-label classification) or not.
When there are 2 classes and you generally have P(c=1) + P(c=0) = 1
then
keras.layers.Dense(2, activation = 'softmax')
keras.layers.Dense(1, activation = 'sigmoid')
both are correct in terms of class probabilities. The only difference being how you supply the labels during training. But
keras.layers.Dense(2, activation = 'sigmoid')
is incorrect in that context. However, it is correct implementation if you have P(c=1) + P(c=0) != 1
. This is the case for multi-label classification where an instance may belong to more than one correct class.