Is there a solution for a distributed grep?
here's the story: I have a bunch of web servers and want to query their application logs (I'm using tomcat, if it matters). I don't want to have to copy the files to a common storage, they are too big, network is espensive and storage is too expensive so I want to keep them on the same web servers. So even haddop+hive or similar solutions won't fly.
So what I'm looking for is perhaps a local agent for which I can send the grep request to all hosts and get the results back.
Anything like that exists?
- No file copying over the network
- No common storage
- Can't install hadoop on the web servers... I just can't...
- grepping text files
Thanks!
BTW, I asked exactly the same question on http://scale.metaoptimize.com/questions/87/distributed-grep but for some reason the server keeps serving me 500s when I try to view this question, so I'm reposting here.