ValueError: Can not squeeze dim[1], expected a dimension of 1, got 3 for 'sparse_softmax_cross_entropy_loss
The error here is from tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits).
The TensorFlow documentation clearly states that "labels vector must provide a single specific index for the true class for each row of logits". So your labels vector must include only class-indices like 0,1,2 and not their respective one-hot-encodings like [1,0,0], [0,1,0], [0,0,1].
Reproducing the error to explain further:
import numpy as np
import tensorflow as tf
# Create random-array and assign as logits tensor
np.random.seed(12345)
logits = tf.convert_to_tensor(np.random.sample((4,4)))
print logits.get_shape() #[4,4]
# Create random-labels (Assuming only 4 classes)
labels = tf.convert_to_tensor(np.array([2, 2, 0, 1]))
loss_1 = tf.losses.sparse_softmax_cross_entropy(labels, logits)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
print 'Loss: {}'.format(sess.run(loss_1)) # 1.44836854
# Now giving one-hot-encodings in place of class-indices for labels
wrong_labels = tf.convert_to_tensor(np.array([[0,0,1,0], [0,0,1,0], [1,0,0,0],[0,1,0,0]]))
loss_2 = tf.losses.sparse_softmax_cross_entropy(wrong_labels, logits)
# This should give you a similar error as soon as you define it
So try giving class-indices instead of one-hot encodings in your Y_Labels vector. Hope this clears your doubt.
If you used Keras' ImageDataGenerator
, you can add class_mode="sparse"
to obtain the correct levels:
train_datagen = keras.preprocessing.image.ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
train_generator = train_datagen.flow_from_directory(
'data/train',
target_size=(150, 150),
batch_size=32,
class_mode="sparse")
Alternatively, you might be able to use softmax_cross_entropy
, which seems to use onehot encoding for the labels.
Changing
loss='sparse_categorical_crossentropy'
to
loss='categorical_crossentropy'
worked for me.