Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Is there a solution for a distributed grep?

here's the story: I have a bunch of web servers and want to query their application logs (I'm using tomcat, if it matters). I don't want to have to copy the files to a common storage, they are too big, network is espensive and storage is too expensive so I want to keep them on the same web servers. So even haddop+hive or similar solutions won't fly.
So what I'm looking for is perhaps a local agent for which I can send the grep request to all hosts and get the results back.
Anything like that exists?

  • No file copying over the network
  • No common storage
  • Can't install hadoop on the web servers... I just can't...
  • grepping text files

Thanks!

BTW, I asked exactly the same question on http://scale.metaoptimize.com/questions/87/distributed-grep but for some reason the server keeps serving me 500s when I try to view this question, so I'm reposting here.

share|improve this question
    
Is there any way to consolidate your logs via syslog or similar, instead of trying to duplicate your log files? –  mfinni Mar 31 '11 at 17:49

4 Answers 4

For simple command runs, such as greping for a specific string on the server's logs, use a parallel SSH client like pssh or dsh.

For more complicated tasks you might want to take a look at MCollective.

share|improve this answer

Might be fun to try this with gnu parallel, maybe something like this:

Put the servers in a file servers.txt. Then:

parallel --sshlogins servers.txt "grep foo logfile"

I haven't tried myself.

share|improve this answer

Why not simply run something like a cron job on your webserver running something like

grep something /path/to/log | ssh -C LogAnalyser "tee >> log_from_host X"

If you want to be able to know when the transfert from all host is finished on LogAnalyser you can start by creating a lock file and delete after tee.

share|improve this answer
    
distribution isn't the goal, it's a constraint. The log files are already on the web servers, now I need to grep them. I can't afford to copy them all to a single host or NAS, so I have to keep them in place and now I want to query them... –  Ran Jul 19 '10 at 6:58
    
Can't you just grep locally ?? –  radius Jul 19 '10 at 7:01
    
Ok I guess I misunderstood... you want to fetch grep result of local file from X server and not to send file over the network to X server and get the result back... I will update my answer –  radius Jul 19 '10 at 7:05
    
if I use the grep and cron technique I need to ssh first to all machines and set up the cron, it's tedious. I want to run ad hoc greps on various files (at the same location on all servers) –  Ran Jul 19 '10 at 17:48

Perhaps something along the following bash loop?

export FQDNS="hostname.domain.tld another.domain.tld"
for host in $FQDNS
do
  ssh $host 'grep "andol was here" /var/log/syslog'
done
share|improve this answer
1  
This will only serially ssh into the remote hosts and run the grep command. You usually want to do that in parallel. –  joschi Jul 19 '10 at 6:33
    
True, so I guess it all depends on what number "a bunch" actually translates into. –  andol Jul 19 '10 at 6:36
    
And how /var/log/syslog comes here ? –  radius Jul 19 '10 at 6:45
    
"a bunch" ~= 50 to 100 –  Ran Jul 19 '10 at 6:59
1  
radius: /var/log/syslog is as much a placeholder as "andol was here". I hardly think this question is about what specific grep command to run at which log file? –  andol Jul 19 '10 at 7:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.