Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am running solr on tomcat server on linux server, when I use the top command I see the "VIRT" very high

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4786 root      20   0 19.6g 3.8g  10m S  200 15.9   8:26.13 java
 1808 root      20   0 6437m 162m 4740 S    2  0.7 101:08.75 java
 4889 root      20   0 19352 1580 1064 R    1  0.0   0:00.08 top
31700 mysql     20   0 6677m 5.2g 4468 S    0 22.0 470:51.49 mysqld
    1 root      20   0 23708 1464  972 S    0  0.0   0:23.06 init

What could be the reason for this or how can I profile this?

share|improve this question

migrated from stackoverflow.com Oct 30 '11 at 12:22

This question came from our site for professional and enthusiast programmers.

add comment

1 Answer

The reason you are seeing such high virtual memory usage is that Solr uses MMapFSDirectory as the default class for manipulating the Lucene index. This class will attempt to map any indexes under Solr control to virtual memory - the more cores/indexes the worse it gets. The fun part is that this is outside the JVM's knowledge/control. The JVM will only report on the (-Xms:128m -Xmx:1024m as an example) min/max memory you specify for your servlet container. It would have been nice for them to warn folks or to use a more conservative directoryFactory as the default.

Change the line in your solrconfig.xml:

directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.StandardDirectoryFactory}"

to point to something like NIOFSDirectoryFactory instead.

share|improve this answer
    
Does NIOFSDirectoryFactory mean that no memory is used for storing indexes? If so, how does this perform in contrast to MMap? –  Peter Kruithof Mar 1 '12 at 9:33
    
@PeterKruithof I would encourage you to visit link and link, and the API for sun.nio.channels.FileChannel - specifically the map() function. Yes, substantial memory is still used, but the difference is that the NIOFS version wraps the a local byte[] with a ByteBuffer and 1) exists within the JVM 2) subject to garbage collection and trouble that can ensue. –  GangstaHog Mar 21 '13 at 17:39
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.