1

Question: How do I fix my servers and get them back to the normal ReplicaSet with no error messages?

I'd offer a serious bounty, but all my points are on SO :(

Details of the Problem

I'm running a 3-node MongoDB ReplicaSet (1 master, 1 slave, 1 arbiter) on separate servers. I was toying around with automatic failover, ensuring that my slave would be promoted to master properly. After I took master down and the slave took over as master, I followed the instructions to re-elect my old master back to primary (Forcing a Member to be Primary). From my new PRIMARY, I ran:

> use admin
> db.auth("admin", "mypassword")
> config = rs.conf()
{
    "_id" : "foo",
    "version" : 1,
    "members" : [
        {
            "_id" : 0,
            "host" : "A",
        },
        {
            "_id" : 1,
            "host" : "B",
        },
        {
            "_id" : 2,
            "host" : "C",
        }
    ]
}
> config.version++
> // the default priority is 1
> config.members[1].priority = 2
> rs.reconfig(config)

This threw an error, so I tried method #2:

db.adminCommand({replSetStepDown:1000000, force:1})

This seemed to botch my servers, causing no server to be promoted to primary and my old master and arbiter to start receiving the following errors:

Mon Aug 29 19:16:22 [rs Manager] replset msgReceivedNewConfig version: version: 2
Mon Aug 29 19:16:22 [rs Manager] replSet info saving a newer config version to local.system.replset
Mon Aug 29 19:16:22 [rs Manager] Server::doWork task:rs Manager exception:unauthorized db:local lock type:2 client:(NONE)
Mon Aug 29 19:16:22 [initandlisten] connection accepted from 192.168.139.50:28130 #476
Mon Aug 29 19:16:22 [initandlisten] connection accepted from 192.168.132.14:32014 #477
Mon Aug 29 19:16:23 [initandlisten] connection accepted from 192.168.139.50:28133 #478
Mon Aug 29 19:16:23 [initandlisten] connection accepted from 192.168.139.50:28136 #479
Mon Aug 29 19:16:23 [initandlisten] connection accepted from 192.168.132.14:32017 #480
Mon Aug 29 19:16:24 [initandlisten] connection accepted from 192.168.139.50:28139 #481
Mon Aug 29 19:16:24 [rs Manager] replset msgReceivedNewConfig version: version: 2
Mon Aug 29 19:16:24 [rs Manager] replSet info saving a newer config version to local.system.replset
Mon Aug 29 19:16:24 [rs Manager] Server::doWork task:rs Manager exception:unauthorized db:local lock type:2 client:(NONE)
Mon Aug 29 19:16:24 [rs Manager] replSet info not electing self, we are not freshest
Mon Aug 29 19:16:24 [initandlisten] connection accepted from 192.168.139.50:28142 #482
Mon Aug 29 19:16:24 [initandlisten] connection accepted from 192.168.132.14:32020 #483
Mon Aug 29 19:16:25 [initandlisten] connection accepted from 192.168.139.50:28145 #484
Mon Aug 29 19:16:25 [initandlisten] connection accepted from 192.168.139.50:28148 #485
Mon Aug 29 19:16:25 [initandlisten] connection accepted from 192.168.132.14:32023 #486
Mon Aug 29 19:16:26 [initandlisten] connection accepted from 192.168.139.50:28151 #487

After some googling, I found this to be a bug with ReplicaSets:

A few bits of information on my setup:

  • I'm running on Mongo 1.8.1
  • I have journal = true
  • I have keyFile = /path/to/file
  • I have user authentication enabled (which appears to be causing the bug)
  • I have a firewall enabled with ports 27017-27019 opened for mongod
  • Everything was working fine prior to the switch
1
  • As you have mentioned, this was a bug you found reported in the Jira ticket. It has been reported fixed in version 1.8.2 and later, as well as the more recent versions. Is this still a problem? Feb 20, 2012 at 6:19

1 Answer 1

1

As the reporter updated in the question, this was reported as a bug, and has since been fixed in a newer release of the software.

Bug reported/fixed here: https://jira.mongodb.org/browse/SERVER-2919

This question should be marked answered/closed.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .