0

The mongo db.adminCommand({top:1}) command returns artifacts from dropped databases. These include the associated collections, as well as some attributes which are not in the database.collection format. The latter will break monitoring tools such as the NewRelic nri-mongodb integration, which currently panics as it cannot parse the value.

The artifacts do not always show when running top from the PRIMARY node, but they will reliably appear when running top from the SECONDARY node.

The issue was first noticed on replica sets running MongoDB server version 3.2.10, and persists as nodes are upgraded to 3.2.21.

Running a repairDatabase on both the PRIMARY and SECONDARY nodes does not remove the artifacts.

Restarting the mongod service on each node will remove the existing artifacts. Using db.dropDatabase() again will create new ones.

To run the top command, I'm using MongoDB shell version v4.0.3, but it also happens via the mgo driver.


Create a new collection via the PRIMARY:

x-mongo-rs:PRIMARY> use drop_test
switched to db drop_test
x-mongo-rs:PRIMARY> db.createCollection("foo")
{ "ok" : 1 }

See it appear normally via top from the SECONDARY:

➜  ~ mongo mongodb-xxx/admin -u xxx -p xxx --quiet --eval 'db.adminCommand({top:1})' | grep drop_test

        "drop_test.foo" : {

Drop the database:

x-mongo-rs:PRIMARY> db.dropDatabase()
{ "dropped" : "drop_test", "ok" : 1 }

See artifacts via top from the SECONDARY:

➜  ~ mongo mongodb-xxx/admin -u xxx -p xxx --quiet --eval 'db.adminCommand({top:1})' | grep drop_test

        "drop_test" : {
        "drop_test.foo" : {

Verify from this node that the database/collection probably shouldn't exist:

x-mongo-rs:SECONDARY> use drop_test
switched to db drop_test
x-mongo-rs:SECONDARY> db.runCommand({collstats:"foo"})
{ "ok" : 0, "errmsg" : "Database [drop_test] not found." }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.