Custom Keras Data Generator with yield
You are using the Sequence API, which works a bit different than plain generators. In a generator function, you would use the yield
keyword to perform iteration inside a while True:
loop, so each time Keras calls the generator, it gets a batch of data and it automatically wraps around the end of the data.
But in a Sequence, there is an index
parameter to the __getitem__
function, so no iteration or yield
is required, this is performed by Keras for you. This is made so the sequence can run in parallel using multiprocessing, which is not possible with old generator functions.
So you are doing things the right way, there is no change needed.
Example of generator in Keras
:
def datagenerator(images, labels, batchsize, mode="train"):
while True:
start = 0
end = batchsize
while start < len(images):
# load your images from numpy arrays or read from directory
x = images[start:end]
y = labels[start:end]
yield x, y
start += batchsize
end += batchsize
Keras wants you to have the infinite loop running in the generator.
If you want to learn about Python generators, then the link in the comments is actually a good place to start.