Programmatically setting properties when using @KafkaListener

The deserializer goes on properties used to create the consumer factory, not the container factory, you can override boot's consumer factory as follows:

@Bean
public ConsumerFactory<?, ?> kafkaConsumerFactory(KafkaProperties properties) {
    Map<String, Object> props = properties.buildConsumerProperties();
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, MyDeserializer.class);
    return new DefaultKafkaConsumerFactory<>(props);
}