NotImplementedError: Cannot convert a symbolic Tensor (2nd_target:0) to a numpy array

I found the solution to this problem:

It was because I mixed symbolic tensor with a non-symbolic type, such as a numpy. For example. It is NOT recommended to have something like this:

def my_mse_loss_b(b):
     def mseb(y_true, y_pred):
         ...
         a = np.ones_like(y_true) #numpy array here is not recommended
         return K.mean(K.square(y_pred - y_true)) + a
     return mseb

Instead, you should convert all to symbolic tensors like this:

def my_mse_loss_b(b):
     def mseb(y_true, y_pred):
         ...
         a = K.ones_like(y_true) #use Keras instead so they are all symbolic
         return K.mean(K.square(y_pred - y_true)) + a
     return mseb

Hope this help!

For me, the issue occurred when upgrading from numpy 1.19 to 1.20 and using ray's RLlib, which uses tensorflow 2.2 internally. Simply downgrading with

pip install numpy==1.19.5

solved the problem; the error did not occur anymore.