I have Ubuntu 9.10 server with 12G Ram/Quad Core/HD 80GB RAID. and i have installed solr lucidworks-enterprise-installer-1.7 on it to index a small database (about 20k articles).
once our editors start to use solr functionalities to search for a specific article (our editors who use solr are 5 editors only), server-load goes up exponentially!!
$ top top - 08:29:29 up 25 days, 20:04, 1 user, load average: 46.23, 59.69, 38.29 Tasks: 2695 total, 1 running, 2693 sleeping, 0 stopped, 1 zombie Cpu(s): 14.2%us, 2.6%sy, 0.0%ni, 45.7%id, 35.9%wa, 0.1%hi, 1.4%si, 0.0%st Mem: 12331880k total, 12245716k used, 86164k free, 3140k buffers Swap: 3229024k total, 3228600k used, 424k free, 139184k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 17448 root 20 0 3710m 1.4g 3300 S 61 11.7 37:32.39 java
EDIT
in my dataconfig.xml
<entity name="root" pk="id" preImportDeleteQuery="data_source:1" query="select a.id, a.body, a.headline title ,a.date datecreated, a.title_id, t.name publisher_name from article as a inner join title as t on t.id=a.title_id" transformer="TemplateTransformer" deltaImportQuery="select a.id, a.body, a.headline title ,a.date datecreated, a.title_id, t.name publisher_name from article as a inner join title as t on t.id=a.title_id where a.id='${dataimporter.delta.id}'" deltaQuery="select id from article where last_update_time > '${dataimporter.last_index_time}'"> <field column="data_source" template="1"/> <field column="data_source_type" template="Jdbc"/> <field column="data_source_name" template="db_solr"/> </entity>
and the table in mysql database
mysql> desc article; +------------------+--------------+------+-----+-------------------+-------+ | Field | Type | Null | Key | Default | Extra | +------------------+--------------+------+-----+-------------------+-------+ | id | int(255) | NO | PRI | | | | nid | int(11) | NO | MUL | 0 | | | title_id | int(11) | NO | MUL | 0 | | | language_id | int(255) | NO | MUL | | | | headline | varchar(255) | YES | | NULL | | | summary | text | YES | | NULL | | | body | text | YES | | NULL | | | author | varchar(255) | YES | | NULL | | | date | date | YES | MUL | NULL | | | parsed_at | datetime | YES | MUL | NULL | | | updated_at | datetime | YES | | NULL | | | last_update_time | timestamp | NO | | CURRENT_TIMESTAMP | | +------------------+--------------+------+-----+-------------------+-------+ 12 rows in set (0.02 sec) mysql> select count(*) from article; +----------+ | count(*) | +----------+ | 19560 | +----------+ 1 row in set (0.00 sec)
also i found 7 instances of the line below by applying ps aux |grep "DlucidworksHome=/etc/solr" | grep -v grep
java -server -DlucidworksHome=/etc/solr -XX:MaxPermSize=256m -DSTOP.PORT=8887 -DSTOP.KEY=stopLucidWorks -Djava.awt.headless=true -Dlog4j.configuration=file:conf/log4j.xml -Dorg.restlet.engine.loggerFacadeClass=org.restlet.ext.slf4j.Slf4jLoggerFacade -Duser.language=en -Duser.country=US -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Djetty.port=8888 -Djetty.home=jetty -jar ./jetty/start.jar ./jetty/etc/jetty.xml ./jetty/etc/jetty-ssl.xml
any idea about what might cause this... (solr takes 11.7% of 12G Ram!!). usually my load average is about 3-5 but once i start solr it becomes 40-70
am i wrong and solr is normal to make such load??
Please excuse my ignorance in solr :)
Thanks for your help