My company has a few servers that we dedicate to running Sphinx Search. All are CentOS 5 with about 48GB of memory and our version of searchd is 1.11. Recently I've been trying to understand why Sphinx uses so little of the available memory for its regular indexes (we don't use RT indexes at the moment). According to the official Sphinx website, all files related to an index except for the .spd and .spp files are stored in RAM. At present this constitutes about 14GG for all our indexes, which can easily fit in RAM. However, when I consult htop for memory usage, it's showing a little less than 1.5GB being used! Interestingly enough, it reports that swap is being used as well, even with vm.swappiness set to 0. If I stop searchd, swap goes down to 0. (It should be noted that the amount of swap used is no greater that the amount of memory being used.) We have it configured so that all our indexes are preopened on startup (preopen_indexes = 1 in our sphinx conf).
I have two questions, given the above information:
- What is Sphinx doing with the data in those indexes, if it isn't keeping it in memory?
- Is it possible to force Sphinx to keep more data in memory?