TensorFlow for binary classification
The original MNIST example uses a one-hot encoding to represent the labels in the data: this means that if there are NLABELS = 10
classes (as in MNIST), the target output is [1 0 0 0 0 0 0 0 0 0]
for class 0, [0 1 0 0 0 0 0 0 0 0]
for class 1, etc. The tf.nn.softmax()
operator converts the logits computed by tf.matmul(x, W) + b
into a probability distribution across the different output classes, which is then compared to the fed-in value for y_
.
If NLABELS = 1
, this acts as if there were only a single class, and the tf.nn.softmax()
op would compute a probability of 1.0
for that class, leading to a cross-entropy of 0.0
, since tf.log(1.0)
is 0.0
for all of the examples.
There are (at least) two approaches you could try for binary classification:
The simplest would be to set
NLABELS = 2
for the two possible classes, and encode your training data as[1 0]
for label 0 and[0 1]
for label 1. This answer has a suggestion for how to do that.You could keep the labels as integers
0
and1
and usetf.nn.sparse_softmax_cross_entropy_with_logits()
, as suggested in this answer.