Setup MongoDB replication set

3 years ago, I’ve setup a huge test environment in my company’s lab, which use mongoDB as the center data storage. At that time, I have no ideas about replication, so I put every thing in one VM. Because lab is managed by our lab admin team, which uses UPS for power failure, and central temperature control. I think this environment 99.99% safe.

However, 1 year ago, the unexpected still happened. Some guys put two many servers in one small chassis, which is too hot and too crowd to blocked cool air flows. The first fire accident happened in our lab. City’s fire brigade came and cut off the lab power supply including UPS, and water filled parts of lab, some servers are totally damaged…

Yes, my central mongoDB server is one of the victim, all of the data are lost…

So from that time, I learned the unexpected will aways happen, so

Don’t put all of your eggs in one basket!”

I began to backup data every week to different servers in different lab in different floor, and learn to use mongo replication.

So what is mongo replica set

From mongo official documentation:

A replica set in MongoDB is a group of mongod processes that maintain the same data set. Replica sets provide redundancy and high availability, and are the basis for all production deployments.

A replica set contains several data bearing nodes and optionally one arbiter node. Of the data bearing nodes, one and only one member is deemed the primary node, while the other nodes are deemed secondary nodes.

The primary node receives all write operations. A replica set can have only one primary capable of confirming writes with { w: "majority" } write concern; although in some circumstances, another mongod instance may transiently believe itself to also be primary. [1] The primary records all changes to its data sets in its operation log, i.e. oplog. For more information on primary node operation, see Replica Set Primary.

The standard replica set deployment is a three-member replica set, and mongo and support up to 50 members, but only 7 voting members, additional members must be non-voting members.

My test environment:

3 VM in different labs

ubuntu 18.04

mongoDB 5.0.0

Change the default mongoDB conf, let mongod listen on all of the interfaces:

 bindIp: 0.0.0.0

And set replication name:

 replSetName: rs

Following is the full configuration for all of three VMs

(Please restart mongod service after this modification):

For the ubuntu environment, make sure you disabled ufw firewall, or you can open default mongo port 27017 open.

Get ufw firewall status

sudo ufw status

Disable ufw

sudo ufw disable

Or just enable 27017 port

sudo ufw allow 27017

And login primary VM(10.124.214.162)’s mongo shell, start this replication:

If all of these services can be accessed, you will get following response:

{ “ok” : 1 }

Then primary mongo cli prompt will change to:

rs:PRIMARY>

And using rs.status() can get more details

If you open the mongo cli on the other two VMs, you will see:

rs:SECONDARY>

Verify the replication

Let’s insert some data in primary node, and check if we can see these data in the secondary mongo instances.

In the primary node:

In any secondary node’s mongo cli, you need to enable cli first:

rs.secondaryOk()

By default cli is enabled on the primary node only:

So data is synced with primary node.

Let’s test if primary dead, if this cluster can auto elect new primary node. In the primary node, just stop mongod service:

systemctl stop mongod

Seconds later, you will found one of secondary node cli prompt updated:

rs:SECONDARY>
rs:PRIMARY>

So this small replica set works as expected.