SQL Server 2005: There is insufficient system memory to run this query

Question

One of our SQL Servers, which has been running stable for quite some time (years), has recently been throwing insufficient memory errors. From the Application Event Log, we see:

Event ID: 701

Description: There is insufficient system memory to run this query.

Our team that manages this server is made up of mostly developers that double up on the sysadmin duties. However, our primary expertise is development. That being said, we are at a loss on how we go about troubleshooting this. We've been scouring forums and whatnot and haven't found anything that matches

So, here are some more details to aid in troubleshooting:

• Our Minimum server memory is set to 0.
• Our Maximum server memory is set to 2000.
• Total Physical Memory is 3,325.85 MB (from sysinfo).
• Total Virtual Memory is 7.10 GB (from sysinfo).
• We were not using AWE to allocate memory, but we are now to see if it makes a difference.
• This error was thrown by a job that was backing up a transaction log, not running a query.
• We have many linked servers. The types of RDBMS on the other side are SQL Server (2005 and 2000), Oracle 10g, and OSI PI systems.
• It is intermittent at this point. We cannot seem to correlate any time or event to the errors.
• Of course, rebooting seems to make it go away for a while, which makes sense due to the nature of the error message.
• This server triples as an application server (a couple of Windows Services) and a web server, as well as the database server.

EDIT:

We are on SP3. Most of the posts we found were pre SP1, which doesn't apply to us.

SELECT  SERVERPROPERTY('productversion'), SERVERPROPERTY ('productlevel'), SERVERPROPERTY ('edition')

returns

9.00.4035.00 SP3 Standard Edition

Answer 1

I would suggest using the -g startup parameter as well. It seems to work for most people and will probably work for you as well. My only concern would be that the underlying issue may not be resolved. For example if there is a memory leak due to a linked server, and the MTL is increased to 512Mb will it just be a longer period between memory issues? I don't know the answer to that but I tend to agree with UndertheFold in that a perfmon may be a good start.

Answer 2

The error message "There is insufficient system memory to run this query." refers to Virtual Address Space (VAS) being unavailable and not memory in conventional sense i.e. within the SQL Server process space.

Given that you are only running with 3GB on this server and SQL Server has been assigned up to 2GB, this means the OS and more importantly anything else on the box has less than 1GB to play with. That's not a whole lot of memory.

If this issue is indeed as the result of a memory leak then it is the VAS outside of the SQL Server process space (memToLeave) that is being consumed.

I would suggest using the -g startup parameter to assign more memory to the memToLeave portion.

See the following article for further information:

http://www.johnsansom.com/sql-server-memory-configuration-determining-memtoleave-settings/

You may also wish to reduce the max memory setting of SQL Server but I would do this as a last resort.

Answer 3

This could be related to a memory leak of a linked server driver, according to this forums thread:

The following is what Microsoft told us.

Apparently processing data using a linked server specifially the fox pro driver causes memory leakage which builds up over time.

Answer 4

This server triples as an application server (a couple of Windows Services) and a web server, as well as the database server.

I would set your min memory - it is quite possible that these other process are "stealing" memory from SQL

You could run a counter log using perfmon to confirm this and/or give yourself more information to identify what the real issue is.

Answer 5

Reference taken from this blog !

There are different alternative to solve this problem.

First, Check your SQL Server setting for “min server memory” and “max server memory”. If you found very small difference in both the value, increase your “max server memory”.

Second, Found out long running queries with its memory usage information and if this query in idle states, please verify and kill this process. The database performance optimization is a major thing for memory usages.

Third, Found out index usages for long running queries because without proper indexing your system DISK I/O increase and its directly affect to your memory.

Fourth, Check the size of the Virtual memory paging file and increase the size of this file.

Fifth, Check the size of the “min memory per query” actually it is by default 1024 KB but in rare situations you can decrease the size of this parameter. Actually, this is not advisable, but you can try it.

Sixth, Try to execute this DBCC command and again this is not advisable because it may affect to the overall performance of the server. But you can try this.