Restore original text from Keras’s imdb dataset

Your example is coming out as gibberish, it's much worse than just some missing stop words.

If you re-read the docs for the start_char, oov_char, and index_from parameters of the [keras.datasets.imdb.load_data](https://keras.io/datasets/#imdb-movie-reviews-sentiment-classification ) method they explain what is happening:

start_char: int. The start of a sequence will be marked with this character. Set to 1 because 0 is usually the padding character.

oov_char: int. words that were cut out because of the num_words or skip_top limit will be replaced with this character.

index_from: int. Index actual words with this index and higher.

That dictionary you inverted assumes the word indices start from 1.

But the indices returned my keras have <START> and <UNKNOWN> as indexes 1 and 2. (And it assumes you will use 0 for <PADDING>).

This works for me:

import keras
NUM_WORDS=1000 # only use top 1000 words
INDEX_FROM=3   # word index offset

train,test = keras.datasets.imdb.load_data(num_words=NUM_WORDS, index_from=INDEX_FROM)
train_x,train_y = train
test_x,test_y = test

word_to_id = keras.datasets.imdb.get_word_index()
word_to_id = {k:(v+INDEX_FROM) for k,v in word_to_id.items()}
word_to_id["<PAD>"] = 0
word_to_id["<START>"] = 1
word_to_id["<UNK>"] = 2
word_to_id["<UNUSED>"] = 3

id_to_word = {value:key for key,value in word_to_id.items()}
print(' '.join(id_to_word[id] for id in train_x[0] ))

The punctuation is missing, but that's all:

"<START> this film was just brilliant casting <UNK> <UNK> story
 direction <UNK> really <UNK> the part they played and you could just
 imagine being there robert <UNK> is an amazing actor ..."

You can get the original dataset without stop words removed using get_file from keras.utils.data_utils:

path = get_file('imdb_full.pkl',
               origin='https://s3.amazonaws.com/text-datasets/imdb_full.pkl',
                md5_hash='d091312047c43cf9e4e38fef92437263')
f = open(path, 'rb')
(training_data, training_labels), (test_data, test_labels) = pickle.load(f)

Credit - Jeremy Howards fast.ai course lesson 5

This encoding will work along with the labels:

from keras.datasets import imdb
(x_train,y_train),(x_test,y_test) = imdb.load_data()
word_index = imdb.get_word_index() # get {word : index}
index_word = {v : k for k,v in word_index.items()} # get {index : word}

index = 1
print(" ".join([index_word[idx] for idx in x_train[index]]))
print("positve" if y_train[index]==1 else "negetive")

Upvote if helps. :)