Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have 10 Petabytes of Microsoft Office documents on an NAS. I want to index it so it can be searched efficiently, but even with a standalone search server on good hardware that'd take well over a year.

Is there a massively parallel indexing solution, or some other way to speed that up dramatically?

share|improve this question

closed as off-topic by joeqwerty, squillman, Ward, cole, Dave M Nov 26 '13 at 18:07

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions seeking product, service, or learning material recommendations are off-topic because they tend to become obsolete quickly. Instead, describe your situation and the specific problem you're trying to solve." – joeqwerty, squillman, Ward, cole, Dave M
If this question can be reworded to fit the rules in the help center, please edit the question.

    
The limiting factor should already be the speed of your NAS, how have you worked out a year? –  JamesRyan Nov 26 '13 at 14:31
    
Experiments indicate that indexing is CPU-bound, not disk io-bound. –  Stephen Bachelor Nov 26 '13 at 15:24
    
Ah, I assumed that when you were asking for features that windows search doesn't do that you were looking more generally at alternatives. –  JamesRyan Nov 27 '13 at 10:00
    
I am looking very generally at alternatives. But any alternative has to produce an efficiently searchable index from Microsoft Office documents. –  Stephen Bachelor Dec 3 '13 at 19:20
    
I'm not looking for product, service, or learning material recommendations. I'm looking for anything that'll quickly produce a searchable index of 10 petabytes of Microsoft Office documents. If that's not on-topic here, what stackexchange site would answer it? –  Stephen Bachelor Dec 9 '13 at 18:27