How to add and remove new layers in keras after loading weights?

You can take the output of the last model and create a new model. The lower layers remains the same.

model.summary()
model.layers.pop()
model.layers.pop()
model.summary()

x = MaxPooling2D()(model.layers[-1].output)
o = Activation('sigmoid', name='loss')(x)

model2 = Model(input=in_img, output=[o])
model2.summary()

Check How to use models from keras.applications for transfer learnig?

Update on Edit:

The new error is because you are trying to create the new model on global in_img which is actually not used in the previous model creation.. there you are actually defining a local in_img. So the global in_img is obviously not connected to the upper layers in the symbolic graph. And it has nothing to do with loading weights.

To better resolve this problem you should instead use model.input to reference to the input.

model3 = Model(input=model2.input, output=[o])

Another way to do it

from keras.models import Model

layer_name = 'relu_conv2'
model2= Model(inputs=model1.input, outputs=model1.get_layer(layer_name).output)