3

So on day 2 of NewJob I've inherited a slight issue with SOLR. The DB team has a number of servers upon which they generate SOLR indexes. The servers aren't beefy enough to run multiple indexes at the same time (mostly since the datasets are gigantic). They want, therefore, a way to determine when a SOLR run starts/ends.

Does anyone know what sequence shows up in the logs for this? I've had the DBs tell me when they kick off runs and when they end, but there don't appear to be any unique log entries at those times that don't appear during times when runs theoretically aren't running (especially as runs must be manually triggered).

Any help would be greatly appreciated.

1 Answer 1

3

Solr's indexes are modified by making API calls to the Solr web service.

Something outside of Solr should be making those calls to manage the members of the Solr index, and that's where you'll need to track the status of the work; Solr itself doesn't have any concept of tracking the progress of a bulk indexing job. You could guess at it badly with some parsing of the access logs, but that's not a good approach.

If you can, dig a little further into how the indexing work is being managed - that'll be where you should focus your efforts.

3
  • 1
    Right now that's a manual process - the DBA triggers the index through the command line. The hope was that there was some way to monitor that in an automated fashion to let users know which servers were/are safe to run SOLR runs on. I don't know that I have a way to monitor the API call in this fashion, unfortunately. Feb 8, 2012 at 22:31
  • @Driftpeasant What exactly are they running on the command line, though? Solr's API driven - something's gotta be talking to the API. Feb 8, 2012 at 22:37
  • Based on your recommendation, I did further investigation - they're kicking it off through one of their custom apps that calls the Solr API. After talking to the Lucerne/Solr dev list, I think it will be easier to add an email from the application when a run starts/ends than trying to parse the logs. Your suggestion is the correct one. Feb 9, 2012 at 19:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .