I have a MongoDB replica set with Three members: primary, secondary and hidden secondary, only for backups. Ideally, I want to add additional instance dedicated to running slow batch operations. The idea is to have this instance run map-reduce style operations which may take very long time to complete, without slowing down the regular queries which are required to finish quickly.

I am not aware of a Mongo configuration that can achieve this. I would love to hear ideas how to handle heavy, offline requests without affecting short, near real time requests.


That's not a matter of configuration. Your normal queries are sent to the current primary. So all you need to do is to make sure that your batch operations (I assume aggregations and/or mapReduce jobs) are run against a secondary. You can do this by setting the readPreference to secondary or secondaryPreferred.

This comes with some serious drawbacks, though:

  1. mapReduce jobs can not will always show their results inline and they can not be saved. A way to mitigate this problem is to save the results from the application side.
  2. Aggregations with an $out stage will be executed on the primary, regardless of the set read preference
  3. Depending on your write concern settings, your secondaries may hold data which is not up to date. This can be a problem in some situations.

In case using a read preference of secondary or secondaryPreferred it's impossible, the only way to minimize the load of long running operations like aggregations our mapReduce jobs is sharding, splitting the load over multiple machines. Note that in order to implement such a load distribution, careful planning and data modeling is required.

| improve this answer | |

Since you are planning to execute slow queries on new replica, you may want to set a delay time proper to your long running map-reduce queries, such as 3 mins or 10 mins. By this delay, you may have the full performance from your new replica. It will not deal multiple queries at once.

Here is the documentation: Delayed Replica Set Members

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.