How to configure a replica set with MongoDB

I would wish to discuss the concept of replication and replica set in MongoDB.

How do we get availability and fault tolerance? And by that we mean if that node goes down, we want to still be able to use the system. And if the primary nodes goes down and we lose it entirely for some reason, let's say there's a lost between backups or a hardware damage making the system unusable. And so what we do to solve both those problems is we introduce replication.

A replica set refers to a set of mongod and mongo nodes that act together and all mirror each other in terms of data. There is one primary and the other nodes are secondaries. But, that selection is dynamic. And the data written to the primary will asynchronously replicate to the secondaries. The application and drivers stay connected to the primary and will and can only write to the primary. Say, if primary goes down, one of the secondaries will perform an election to elect a new primary. To elect a new primary, we have to have a strict majority of the original number of nodes. So, since the original number of nodes here was 3, we need 2 nodes to elect a new primary and that's the number we have. So, if one went down, then anyone else can become primary. And in that case, the app will connect to the primary for the rights, through the driver. All transparently.

Later if the down servers gets up, it will join the replica set as a secondary. And the minimum number of nodes is 3 - because if we have less than 3, then what would remain would not be a majority of that set of the original set. And so, there would be no way to elect the new primary. So, we would go with no primary, which means we could no longer take rights.

You can actually follow the MongoDB Manual to setup your replica set. It's pretty clear. Here are some critical steps described below:

1. Change your configuration file and add the following line.Don't mess it up with master/slave replication, because replica set is meant to be a replacement of master/slave replication. So you may want to remove those master/slave related config from your configuration files.

   replSet = [set name]
   

   EDIT: The replSet does not seem to exist anymore in recent versions of mongoDB, at least it is no longer documented. The following seems to have done the trick in my case.

   replication: 
     replSetName: "smm"
   

2. Restart your mongod instance:

   systemctl restart mongodb
   // or
   service mongod restart
   

3. go to local database and initiate your replica set. Don't pass anything to the initiate function, and mongodb will handle everything just well. Note it will use your hostname as the name of current instance and as I know it's not that easy to change. So you may want to change your host name before doing so.

   use local
   rs.initiate()
   

That's it. Your set is good to go. If you have other member to join the set, you need to do the 1/2 steps, and go to your primary instance and type:

rs.add("hostname:port")

Only when you want to change configs of replica set, do you need to type:

var conf = rs.conf();
// change your conf here
rs.reconfig(conf);

Note this will lead to server offline a little bit time. If you are doing it online, be careful.