4

I am running solr on tomcat server on linux server, when I use the top command I see the "VIRT" very high

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4786 root      20   0 19.6g 3.8g  10m S  200 15.9   8:26.13 java
 1808 root      20   0 6437m 162m 4740 S    2  0.7 101:08.75 java
 4889 root      20   0 19352 1580 1064 R    1  0.0   0:00.08 top
31700 mysql     20   0 6677m 5.2g 4468 S    0 22.0 470:51.49 mysqld
    1 root      20   0 23708 1464  972 S    0  0.0   0:23.06 init

What could be the reason for this or how can I profile this?

1 Answer 1

3

The reason you are seeing such high virtual memory usage is that Solr uses MMapFSDirectory as the default class for manipulating the Lucene index. This class will attempt to map any indexes under Solr control to virtual memory - the more cores/indexes the worse it gets. The fun part is that this is outside the JVM's knowledge/control. The JVM will only report on the (-Xms:128m -Xmx:1024m as an example) min/max memory you specify for your servlet container. It would have been nice for them to warn folks or to use a more conservative directoryFactory as the default.

Change the line in your solrconfig.xml:

directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.StandardDirectoryFactory}"

to point to something like NIOFSDirectoryFactory instead.

2
  • Does NIOFSDirectoryFactory mean that no memory is used for storing indexes? If so, how does this perform in contrast to MMap? Mar 1, 2012 at 9:33
  • @PeterKruithof I would encourage you to visit link and link, and the API for sun.nio.channels.FileChannel - specifically the map() function. Yes, substantial memory is still used, but the difference is that the NIOFS version wraps the a local byte[] with a ByteBuffer and 1) exists within the JVM 2) subject to garbage collection and trouble that can ensue.
    – GangstaHog
    Mar 21, 2013 at 17:39

You must log in to answer this question.