Kubernetes: how to scale my pods
TL;DR: You need to scale your deployment instead of the replica set directly.
If you try to scale the replica set, then it will (for a very short time) have a new count of 5. But the deployment controller will see that the current count of the replica set is 5 and since it knows that it is supposed to be 3, it will reset it back to 3. By manually modifying the replica set that was created for you, you are fighting with the system controller (which is untiring and will pretty much always outlast you).
This is working for me
kubectl scale --replicas=<expected_replica_num> deployment <deployment_label_name> -n <namespace>
Example
# kubectl scale --replicas=3 deployment xyz -n my_namespace