Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm having a problem with virtual memory usage using MongoDB on a server. NOTE: I have only a local user account without root privs.

The server's VM limit (from ulimit -v) is 120MB.

With an empty database, MongoDB is reporting 100MB of virtual memory used.

> db.serverStatus().mem 
{
    "bits" : 32,
    "resident" : 23,
    "virtual" : 100,
    "supported" : true,
    "mapped" : 0
}

As soon as I try to create my first data entry (through db.addUser in the admin database), it's failing with an out of memory error.

done allocating datafile DBPATH/admin.0, size: 16MB,  took 0.244 secs

ERROR:   mmap() failed for DBPATH/ admin.0 len:16777216 errno:12 Cannot allocate memory

I understand that with a small VM limit and 32-bit mongodb, I'm not going to be able to create gigantic databases, but that's OK. I don't need gigantic databases.

What I can't understand is why it is using so much virtual memory out of the gate, and why it can't create a single 16MB database table.

I have read the accepted answer on this other question. It clarified how mongodb uses virtual memory, but it didn't help me to understand what's going wrong here.

share|improve this question
add comment

closed as off topic by MDMarra, Nils, mdpc, RolandoMySQLDBA, Khaled Mar 8 '13 at 16:33

Questions on Server Fault are expected to relate to professional server, networking, or related infrastructure administration within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

2 Answers

up vote 4 down vote accepted

You need to provide adequate resources for your MongoDB server -- that means LOTS of RAM.
You are limiting it to virtually none, so you should not expect it to work unless you set more reasonable memory use restrictions.


You seem to be laboring under the mistaken assumption that 100MB is a large amount of memory.
It's not.

A simple shell (let's take plain old vanilla Bourne sh as an example) is nearly 10MB Virtual Size.
A more complicated shell (say the Bourne-Again Shell bash) is just under 25MB.
An Apache web server worker can have a VM size of 150MB or more.

A database engine, even a NoSQL one, that is only 100MB virtual size and communicates over the network is frankly a programming miracle. If you expect to be able to do anything with that miracle you're going to have to give it enough RAM to actually hold and work with data.

share|improve this answer
    
I appreciate the info but could have done without the snark. I am aware that 100MB is not a lot by modern standards, but MongoDB promises that the virtual memory usage will be: "slightly larger than mapped" in typical installations, and recommends sizing your RAM requirements based on your working data set. Asking why the VM use was nearly ten times the data set size did not seem to be an unreasonable question. –  Lynn Mar 7 '13 at 19:19
    
The key words there are "in typical installations", which you're quite clearly not one of. –  ceejayoz Mar 7 '13 at 19:20
1  
No snark intended, @Lynn. I've no way of gauging your experience level, but the fact that you seem surprised at a 100MB virtual size for an unloaded DB server lead me to think the illustrative examples might be useful - if not to you, then perhaps to the next person who comes along wondering the same thing. (It would also probably be more reasonable for Mongo to say the resident size will be slightly larger than mapped - in your case the VM size is bounded more by shared libraries than your data set.) –  voretaq7 Mar 7 '13 at 19:38
    
@voretaq7 - Apologies if I misread your answer. Thank you for responding. –  Lynn Mar 7 '13 at 19:41
add comment

The virtual memory usage will represent the entire size of the data files (including empty space), not just the data in the database. When you call mmap() on a file, it just maps that all into virtual memory - it does not care what's in the file.

If you have a low virtual memory limit, then that host is simply not going to be suitable for MongoDB beyond a very small data set. You will be helped by running with the smallfiles option, but only a little.

Working set and resident memory are a completely different thing to the virtual memory requirements for MongoDB. For virtual memory, you will need to have at least the total size of your data files available, and if you run with journaling (you should, but it's generally not possible with 32-bit) that requirement doubles. As a general statement you should have a host with unlimited virtual memory to run MongoDB, if there are limitations imposed it's probably going to cause you a lot of problems.

It's not part of what you are asking, but I would also generally not recommend using the 32-bit version of MongoDB for anything besides some basic testing.

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.