Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

What is the most efficient way to perform full-text searches on files in a filesystem?

share|improve this question
... "Binarise"? – Ignacio Vazquez-Abrams Oct 23 '10 at 2:56
"Compile" may be a better word – user58006 Oct 23 '10 at 2:59
Not really. Index might be. Brute-force search, use your OS's find, grep, etc. If it's a structured file, consider loading it into a database engine like SQLite and then working on it. – TessellatingHeckler Oct 23 '10 at 3:00
This sounds more like a question for Super User of Stack Overflow. – Cristian Ciupitu Oct 23 '10 at 3:02
This question is overly broad. You should narrow its scope and be more specific. Otherwise find ... -print0 | xargs -0 grep ... or find ... -exec grep ... + are the most general-purpose answers that are efficient enough. Specifying the starting directory and the proper options will do a lot for efficiency. Indexed searches are very fast, but the index has to exist first. It all depends on what you're trying to do. – Dennis Williamson Oct 23 '10 at 5:16

If the number of queries greatly exceeds the number of changes to the files then the most efficient way to perform full-text search is to use a pre-built index that gets updated when files are added, changed or removed.

The type of index needed depends on the type of searches. For example, do you search on words or phrases? Do you want to take into account synonyms? Do you search on word-fragments?

What kinds of efficiencies are you looking for? Development time efficiency? Efficiencies in software licencing costs? CPU time efficiency? Disk IO efficiency? The way you weight these will have a large effect on which solution is optimal for you.

See also Wikipedia

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.