LSTM Followed by Mean Pooling
I just attempted to implement the same model as the original poster, and I'm using Keras 2.0.3
. The mean pooling after LSTM worked when I used GlobalAveragePooling1D
, just make sure return_sequences=True
in the LSTM layer. Give it a try!
Adding TimeDistributed(Dense(1))
helped:
sequence = Input(shape=(max_sent_len,), dtype='int32')
embedded = Embedding(vocab_size, word_embedding_size)(sequence)
lstm = LSTM(hidden_state_size, activation='sigmoid', inner_activation='hard_sigmoid', return_sequences=True)(embedded)
distributed = TimeDistributed(Dense(1))(lstm)
pool = AveragePooling1D()(distributed)
output = Dense(1, activation='sigmoid')(pool)