We have two IIS 7 servers that run in web garden. There's a copy of every site on each server. Files are synchronized using DFS (only source files). Images & big files are located on different server using share.
Atleast few time a week, we get error
"Not enough storage available" when reading any file or processing a server code. IIS Server is unable to map path to a local disk (not just the one that synchronizes with DFS), there also seems to be a problem loading group policy permissions (we get various errors) and we can't open any share on any server on lan. There are also I/O errors when reading files from other custom applications. Disk speed and latency during this fall-out are both ok. If we try to copy a file or test a drive speed, usually works. Sometimes, we can't even run "my computer" or any other software (for example performance monitor). It can only start if we "run as administrator". There's more than enough storage available on every disk.
Most of the sites stop working (but not all). After server restart, everything works ok for a few days and then the problem reoccurs.
Servers are virtualized using VMWARE, there are 50 IIS pools (some are mem limited) and around 200 running sites on each server. Both servers have 16GB of RAM (only about 80% used). CPU is constantly between 30 - 60%.
We tried changing the IRPStackSize registry property to a higher number, but there were no changes. Problems still exists. We tried to disable antivirus (NOD) – no success.
Any hint? Maybe we are close to IIS limits (to many sites/pools)? Maybe 32bit OS and 16GB of RAM? Maybe there is some 'secret' registry settings... We don't think that everyday restart is long-term solution.