Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm looking for a tool to find a line containing a searced text inside a file that has a size of 4 GB

share|improve this question
1  
Is this for a one off search or will you have to do it repeatedly? Is the search string always the same or can it vary? –  davey Jan 8 '10 at 18:27
    
One search only –  Paul Jan 8 '10 at 18:31

8 Answers 8

up vote 3 down vote accepted

When you use *nix, you could also use split to get smaller files and then parse it with whatever you like, be it grep, awk, ...

share|improve this answer

If you have powershell installed you could use

select-string -pattern <your_string> -path <path_to_file>

It probably won't be fast, but won't choke like find or findstr probably will.

share|improve this answer

Use grep. Nothing will be any fast.

share|improve this answer

Not free, but if this is a text file, then BareTailPro might do the job. I have used it to search for text in logfiles which were too large to fit in memory. One advantage is that it does not just show the text, but allows you to jump into the text file, so you can see the lines before and after the hits.

share|improve this answer

If you do it often enough, and the file is broken into logical lines, you could load it into Splunk and search from there. It'll do indexing for you to be quick(er).

share|improve this answer

I'd use grep on *nix, and I'd use a higher end text editor (e.g Notepad++) on Windows.

share|improve this answer
    
I tried opening a 2 GB CSV in Notepad++ and it blew up nicely :-) –  Chris_K Jan 8 '10 at 20:53

It depends on the application, the response time needed, and what you're willing to do to meet those goals.

Recently, I was working with a 10+ GB, 50+ million line text file and had a need to search for specific strings in each line. Standard Unix tool "grep" did the trick, but took an unacceptably long time (multiple minutes). I imported the text into a postgreslq DB (it was a CSV file, easily imported), and once indexed on the key I needed to search on, it took under 1 second to find my record.

Granted, my workstation is single-core, with only 4GB RAM, a 4-year-old 2GHz CPU, and a top-heavy filesystem (ZFS) using 5+ year-old consumer PATA drives. Your mileage will certainly vary. Still, the time difference between the two methods is staggering.

If your data is free-form text, you might still consider importing into a DB which supports full-text search and indexes appropriately to support such searches.

Even if you have the RAM to have the entire file cached and a fast machine, doing a linear search of files this size will be time inefficient, depending (once again) on the application.

share|improve this answer

At the Windows CMD prompt, there are two commands FIND and FINDSTR. They will probably choke on a file that size or be very slow, but you already have them.

Type help findstr and help find for documentation.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.