No, this is not possible, you cannot run multiple instances using the same files at present - the key piece of functionality you would need (managing which instance has the ability to write to a file) does not exist. I don't think this is on the list of feature requests either (I could not find it), and given the number of potential issues I can think of allowing this to be done, it would seem like a long shot in terms of a request, but you are welcome to request it.
The $or
query example you list (and you suggest you have more complicated ones) is going to run multiple queries in parallel, and by the looks of it you then have essentially nested logical $or
s by listing multiple _id
in each clause. With multiple scans for each of the clauses in the $or
, even with a covered index query that is still going to be a large number of scans of the index when that array is large.
If you are not using covered indexes (look for indexOnly to be true
in your explains), then that would mean a lot of document scanning as well and unless your entire data set fits into memory, that is going to mean a lot page faulting.
Since you already state that this is the "only way" to do this on your system (I think a schema review would also be a good idea), then currently, if you are running into CPU issues on a single host, then replication or sharding are your two options to scale out horizontally. I would also make sure that the CPU is in user land, not system (the easiest way to do this is install MMS with munin-node and track user (usually mongod if its a dedicated system) versus system CPU over time.
Before you do that though, make sure you are running on 2.2 - one of the major improvements in 2.2 was the switch to TCMalloc - I cannot be certain, because malloc issues can be hard to diagnose/define at the best of times, but if you are running 2.0, TCMalloc may help you here.