How to change the number of replicas of a Kafka topic?
You can also use kafkactl for this:
# first run with --validate-only to see what kafkactl will do
kafkactl alter topic my-topic --replication-factor 2 --validate-only
# then do the replica reassignment
kafkactl alter topic my-topic --replication-factor 2
Note that the Kafka API that kafkactl is using for this is only available for Kafka ≥ 2.4.0.
Disclaimer: I am contributor to this project
To increase the number of replicas for a given topic you have to:
1. Specify the extra replicas in a custom reassignment json file
For example, you could create increase-replication-factor.json and put this content in it:
{"version":1,
"partitions":[
{"topic":"signals","partition":0,"replicas":[0,1,2]},
{"topic":"signals","partition":1,"replicas":[0,1,2]},
{"topic":"signals","partition":2,"replicas":[0,1,2]}
]}
2. Use the file with the --execute option of the kafka-reassign-partitions tool
[or kafka-reassign-partitions.sh - depending on the kafka package]
For example:
$ kafka-reassign-partitions --zookeeper localhost:2181 --reassignment-json-file increase-replication-factor.json --execute
3. Verify the replication factor with the kafka-topics tool
[or kafka-topics.sh - depending on the kafka package]
$ kafka-topics --zookeeper localhost:2181 --topic signals --describe
Topic:signals PartitionCount:3 ReplicationFactor:3 Configs:retention.ms=1000000000
Topic: signals Partition: 0 Leader: 2 Replicas: 0,1,2 Isr: 2,0,1
Topic: signals Partition: 1 Leader: 2 Replicas: 0,1,2 Isr: 2,0,1
Topic: signals Partition: 2 Leader: 2 Replicas: 0,1,2 Isr: 2,0,1
See also: the part of the official documentation that describes how to increase the replication factor.