Keras keyboard interrupt to stop training?

You could catch the KeyboardInterrupt exception and save the model within the except block:

save_path = './keras-saves/_latest.ckpt'
try:
    model.fit(x_train, y_train,
              batch_size=batch_size,
              epochs=epochs)
except KeyboardInterrupt:
    model.save(save_path)
    print('Output saved to: "{}./*"'.format(save_path))

Best way i found is to use mouse position on screen as input.

In the following example, if you move your mouse to left edge (x<10) keras will stop:

def queryMousePosition():
    from ctypes import windll, Structure, c_long, byref
    class POINT(Structure): _fields_ = [("x", c_long), ("y", c_long)]
    pt = POINT()
    windll.user32.GetCursorPos(byref(pt))
    return pt.x, pt.y  # %timeit queryMousePosition()


class TerminateOnFlag(keras.callbacks.Callback):
    def on_batch_end(self, batch, logs=None):
        mouse_x, mouse_y = queryMousePosition()
        if mouse_x < 10:
            self.model.stop_training = True

callbacks=[keras.callbacks.ReduceLROnPlateau(), TerminateOnFlag()]

model.fit_generator(..., callbacks=callbacks, ...)

(You can easly add different kind of online interactions with the mouse position as input...)